Skip to content

fix: allow exact remainders with zero precision - #156

Open
Shubham-Padkonde wants to merge 1 commit into
cockroachdb:masterfrom
Shubham-Padkonde:fix/unlimited-precision-remainder
Open

Shubham-Padkonde wants to merge 1 commit into
cockroachdb:masterfrom
Shubham-Padkonde:fix/unlimited-precision-remainder

Conversation

@Shubham-Padkonde

Copy link
Copy Markdown

Fixes #134.

BaseContext.Rem(New(100, 100), New(10, 1)) currently returns DivisionImpossible even though the remainder is exactly zero. The quotient digit-count check treats the default precision of zero as a zero-digit limit, while the context uses zero to disable rounding.

Apply the quotient-size limit only when precision is nonzero, and document that behavior for Rem. Explicit precision limits and the existing special-value handling remain in place.

Regression coverage includes the reported large-exponent case, a large integer coefficient, fractional and negative operands, signed zero, and both result/operand aliases. A separate test checks the failing and successful sides of an explicit precision limit.

Validation on Windows amd64, Go 1.27.1:

  • All 21 unlimited-precision regression subcases failed before the fix and pass afterward.
  • go test -count=1 ./... passes.
  • go vet -unsafeptr=false ./... passes.
  • go test -run='^$' -bench=. -benchtime=1x ./... passes.
  • git diff --check passes.

The Linux, race, older-Go, and ARM CI matrix has not been run locally.

Prepared with Codex assistance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Context.Rem fails for Rem(100E100, 10)

1 participant