Skip to content

Get rounding closer to Excel - #95

Merged
borgar merged 2 commits into
masterfrom
rounding
Oct 16, 2026
Merged

borgar merged 2 commits into
masterfrom
rounding

Conversation

@borgar

@borgar borgar commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

Number formats now round in two steps, matching Excel: first the exact value is rounded to 15 significant digits, with a tie going toward zero; then that decimal is rounded to the pattern's last digit, with a tie going away from zero. Digits are always written out in full, so large and small values no longer appear as e+21 or similar.

  • 1.005 with 0.00 gives 1.01 (was 1.00)
  • 2 ** 70 with #,##0 gives 1,180,591,620,717,410,000,000 (was garbled)
  • Engineering notation moves the exponent when rounding carries: 999.96 with ##0.0E+0 gives 1.0E+3
  • General rounds a tie at the tenth digit up, as Excel does

This is an alternative to #93, using a faster, arithmetic-based approach.

Notable differences from #93

Engineering notation when rounding carries:

Pattern Value #93 This PR
##0.0E+0 999.96 1000.0E+0 1.0E+3
##0.0E+0 999960 1000.0E+3 1.0E+6
##0.0E+0 999999999999999 + 0.5 1000.0E+12 1.0E+15

General with a tie at the tenth digit:

Value #93 This PR
4.2394791745 4.239479174 4.239479175
19147467.115 19147467.11 19147467.12
42299445.995 42299445.99 42299446

This is because round() now follows the same rules as the formats:

Call master This PR
round(1.005, 2) 1 1.01
round(0.285, 2) 0.28 0.29
round(1e15 + 5) 1000000000000005 1000000000000000

Vulgar fractions keep the old rounding, which matches Excel more closely. A known difference is recorded as an expected-failure test.

@borgar
borgar merged commit 8dd98c9 into master Oct 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant