Repository navigation
Conversation
A fixed-point pattern rounded the number with Math.round(number * 10 ** places), and wrote its digits with String(), so: - a number of 1e21 or more came out garbled: `#,##0` of 1e300 gave `1e+,300` - so did a small number with many decimals: `0.0000000000` of 1e-7 gave `0.0000000000` - rounding followed the double's binary value, not Excel: `0.00` of 1.005 gave `1.00` (Excel: `1.01`), and `0` of 10^15+5 gave `1000000000000005` (Excel: `1000000000000000`) Excel rounds in two steps: first the double's exact value to 15 significant digits, a tie going toward zero; then that decimal to the pattern's last digit, a tie going away from zero. toDecimal does the first step: toPrecision(15) rounds the exact value, and a tie, which it rounds the wrong way, is detected exactly with float arithmetic. roundDecimal does the second step on the digit string, and the integer and fraction digits come out of it as strings, so no number is formatted by String() any longer. The exponent pattern's check for a significand rounding up to 10 now uses the same rounding, so that it agrees with the digits written. Number.MAX_SAFE_INTEGER under `0` now gives 15 significant digits, as `0.0` already did.
An exponent pattern computed its significand as number * 10 ** -exp in floating point and then rounded that, so its digits could differ from the number's own: `0.00000000000000E+00` of 3 * 2 ** 51 + 1 gave `6.75539944105575E+15` (Excel: `6.75539944105574E+15`), and `0.00E+00` of 1.005 gave `1.00E+00`. The significand now takes the number's 15-digit decimal with its point shifted, and the exponent comes from that decimal's point, so both of Excel's rounding steps apply as they do for fixed point. Subnormal numbers, which Excel doesn't have, keep their shortest representation (`0.000E+000` of 2.33e-321 stays `2.330E-321`) rather than show the digits of their exact value.
The tie branch of toDecimal has a second test for a value whose 15-digit rounding carries into a new integer digit; no test reached it. Exponent patterns of a subnormal number had the exponent off by one (Math.log10 of 1e-320 floors to -321), which the branch fixes without a test.
The shortest representation of a subnormal can have 16 digits, so the 15-digit rounding applies to it; the comment said it is used as is.
gthb
marked this pull request as ready for review
October 15, 2026 14:44
gthb
changed the title
Excel's TEXT(501.5,"0.00E+00") is 5.02E+02; master gave 5.01E+02.
borgar
added a commit
that referenced
this pull request
Oct 16, 2026
Number formats now round in two steps, matching Excel: first the exact value is rounded to 15 significant digits, with a tie going toward zero; then that decimal is rounded to the pattern's last digit, with a tie going away from zero. Digits are always written out in full, so large and small values no longer appear as `e+21` or similar. - `1.005` with `0.00` gives `1.01` (was `1.00`) - `2 ** 70` with `#,##0` gives `1,180,591,620,717,410,000,000` (was garbled) - Engineering notation moves the exponent when rounding carries: `999.96` with `##0.0E+0` gives `1.0E+3` - `General` rounds a tie at the tenth digit up, as Excel does This is an alternative to #93, using a faster, arithmetic-based approach. ### Notable differences from #93 Engineering notation when rounding carries: | Pattern | Value | #93 | This PR | |---|---|---|---| | `##0.0E+0` | `999.96` | `1000.0E+0` | `1.0E+3` | | `##0.0E+0` | `999960` | `1000.0E+3` | `1.0E+6` | | `##0.0E+0` | `999999999999999 + 0.5` | `1000.0E+12` | `1.0E+15` | `General` with a tie at the tenth digit: | Value | #93 | This PR | |---|---|---| | `4.2394791745` | `4.239479174` | `4.239479175` | | `19147467.115` | `19147467.11` | `19147467.12` | | `42299445.995` | `42299445.99` | `42299446` | This is because `round()` now follows the same rules as the formats: | Call | `master` | This PR | |---|---|---| | `round(1.005, 2)` | `1` | `1.01` | | `round(0.285, 2)` | `0.28` | `0.29` | | `round(1e15 + 5)` | `1000000000000005` | `1000000000000000` | Vulgar fractions keep the old rounding, which matches Excel more closely. A known difference is recorded as an expected-failure test. Co-authored-by: Gunnlaugur Thor Briem <gunnlaugur@gmail.com>
Owner
|
Closed by #95 which is loosely based on this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fix how
format()rounds and writes numbers in fixed-point and exponent patterns:#,##0of 2^70 was1,.18,059,162,071,741,13e,+21and is now1,180,591,620,717,410,000,000.0.0000000000of 1e-7 was0.0000000000and is now0.0000001000.0.00E+00of 1e-320 was1.00E-321and is now1.00E-320.TEXT:0.001.011.000.000.290.2803201000000000000000100000000000000501234567890123451234567890123460.00000000000000E+006.75539944105574E+156.75539944105575E+15Why
Excel rounds a number for display in two steps:
Before this change, numfmt mismatched that in two ways:
number * 10 ** placeswithMath.round, which works on the double's binary valueString(), which switches to exponent notation outside 1e-6 to 1e21How
lib/toDecimal.ts:toDecimaldoes step 1:toPrecision(15), which rounds a tie away from zero, plus an exact float-arithmetic test for a tie, which then goes the other way.roundDecimaldoes step 2 on the digit string.0ofNumber.MAX_SAFE_INTEGERis now9007199254740990(15 significant digits), as0.0already wrote it.In a quick check I see no slowdown, indeed this seems a little faster: formatting 1e6 numbers with
#,##0.00took 6–16% less time than on master (Node 24, two runs).