Skip to content

Add fixture-backed differential tests, benchmarks, squaring optimization, and fix G2 decompression#71

Merged
MatteoMer merged 11 commits into
mainfrom
zolt-arith-fixtures-research
Apr 5, 2026
Merged

Add fixture-backed differential tests, benchmarks, squaring optimization, and fix G2 decompression#71
MatteoMer merged 11 commits into
mainfrom
zolt-arith-fixtures-research

Conversation

@MatteoMer

@MatteoMer MatteoMer commented Apr 5, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

Implements the full P0 + P1 tiers of the zolt-arith test & benchmark matrix: fixture-backed differential testing across all arithmetic modules, a shared microbenchmark harness, dedicated ARM64/x86-64 Montgomery squaring, and a bugfix for G2 point decompression.

Fixture-backed tests (P0 complete)

  • Field: BN254 scalar arithmetic and serialization vectors under src/testdata/field/
  • Transcripts: RFC 7693 BLAKE2b vectors + Jolt challenge/state compatibility vectors
  • Accumulators: batch inverse, reduceMulU64/U128, sumOfProducts fixtures
  • MSM: G1 Fr and G1 i128 small/medium vectors with arkworks-validated outputs
  • Pairing: generator scalar pairing vectors with exact Fp12 byte comparison

Fixture-backed tests (P1 complete)

  • Extension fields: Fp2/Fp6/Fp12 differential fixtures (add/sub/mul/square/inv/frobenius/cyclotomic_square)
  • G2 curve ops: scalar_mul, add, double, neg — 16 vectors against arkworks
  • Point compression: G1/G2 compress→decompress roundtrip with arkworks coordinates
  • GLV: G1/G2 scalar multiplication verified against arkworks oracle (10 vectors each)
  • G2 MSM: extended from [1,2,4,8] to [1,2,4,8,16,32] points; first tests in g2_msm.zig
  • Dory commitment: replicates Zolt SRS hash-to-curve in Rust, verifies commitment for nv=2,3,4
  • GPU field crossover: CPU-computed expected values at sizes 4, 16, 64

Arkworks differential tooling

  • tools/zolt-arith-diff/: Rust generator producing fixture corpus from arkworks-bn254
  • testdata/zolt-arith-diff/: checked-in differential vectors (field, accumulator, MSM, pairing, transcript, extensions, g2, glv, point_compression, dory, gpu)
  • tools/zolt-arith-diff/check.zig: Zig-side differential verifier (~21 test blocks)

Shared microbenchmark harness

  • bench/zolt_arith/bench_harness.zig: zero-dep harness with multi-sample stats (min/median/mean/p99/stddev), auto-scaled time units, structured [BENCH] output
  • Refactored field_micro.zig to use the harness (same 8 ops for Fp and Fr)
  • New pairing_micro.zig: pairingFp, millerLoop, millerLoopPrepared, finalExponentiation
  • Build steps: zig build bench-zolt-arith-field, zig build bench-zolt-arith-pairing

Dedicated Montgomery squaring (ARM64 + x86-64)

  • arm64MontgomerySquare256: exploits a[i]*a[j] symmetry — 10 multiplies (6 cross + 4 diagonal) vs 16 in generic mul
  • montgomerySquareX86: same algorithm using ADX dual carry chains (mulxq/adcxq/adoxq)
  • ~5% speedup on Fp.square (17.1ns vs 18.1ns on Apple Silicon)

Bug fix: G2 point decompression (fp2Sqrt)

  • decompressG2 returned null for every non-identity G2 point (including the generator)
  • Root cause: the fp2Sqrt fallback branch negated a guaranteed quadratic residue, converting it into a guaranteed non-residue (since p ≡ 3 mod 4 for BN254, negation flips QR status). It also swapped the roles of c (real) and d (imaginary).
  • Fix: unified both branches to solve for c via c² = (a±t)/2, then derive d = b/(2c)
  • G2 compression tests now verify full compress→decompress roundtrip including y-coordinates

Test plan

  • zig build test — all package tests pass (160+)
  • zig build test-zolt-arith-diff — all 21 differential test blocks pass
  • zig build bench-zolt-arith-field produces [BENCH] output
  • zig build bench-zolt-arith-pairing produces [BENCH] output
  • G2 generator roundtrips through compress→decompress
  • fp2Sqrt verified for G2 generator, [2]G, [42]G, zero, pure-real, pure-imaginary
  • x86-64 squaring needs validation on an x86 machine

🤖 Generated with Claude Code

@MatteoMer MatteoMer self-assigned this Apr 5, 2026
@MatteoMer MatteoMer changed the title Zolt arith fixtures research Zolt arith fixtures tests Apr 5, 2026
@MatteoMer MatteoMer changed the title Zolt arith fixtures tests Add fixture-backed differential tests, microbenchmark harness, and ARM64 squaring optimization Apr 5, 2026
@MatteoMer MatteoMer force-pushed the zolt-arith-fixtures-research branch from 77e2fec to 5863fb5 Compare April 5, 2026 17:43
@MatteoMer MatteoMer changed the title Add fixture-backed differential tests, microbenchmark harness, and ARM64 squaring optimization Add fixture-backed differential tests, benchmarks, squaring optimization, and fix G2 decompression Apr 5, 2026
MatteoMer and others added 11 commits April 5, 2026 21:30
@MatteoMer
Add zolt-arith fixture-backed tests
387395c
@MatteoMer @claude
Add arkworks-based fixture generation and cross-validation tooling fo…
f1d23e3 
…r zolt-arith

Introduces a differential testing pipeline: an arkworks Rust tool generates
reference fixtures (field ops, MSM, pairing), and a Zig checker validates
zolt-arith results against them. Includes micro-benchmarks and test matrix docs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@MatteoMer @claude
Add accumulator and transcript differential fixtures (P0)
9f987f2 
Extend the arkworks-based differential testing infrastructure with the
two remaining P0 gaps from the test/benchmark matrix:

Accumulators: add fixtures and Zig verification for sumOfProducts,
batchInverse, mulU64 (Barrett path), and mulU128 (Barrett path).

Transcripts: add an independent Rust Blake2bTranscript oracle and
fixtures for state vectors and challenge vectors, covering the public
API (appendLabel, appendU64, appendScalar, challengeU128,
challengeScalar128Bits) including multi-step sequences.

All 11 differential tests pass via `zig build test-zolt-arith-diff`.
Package tests (135/135) are unaffected.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@MatteoMer @claude
fix: enable blake2b transcript test discovery and correct stale vectors
ce36b51 
The 19 blake2b transcript tests were never discovered by the test
runner because transcripts/mod.zig only imported blake2b.zig as a
pub const, not in a test block. Add the missing test reference.

The 4 inline "jolt compatibility" tests and the state vector fixture
file contained stale expected values that did not match standard
Blake2b-256 output. Update them to the correct values verified against
Python hashlib and the Rust differential oracle.

Package test count: 135 → 155 (20 blake2b tests now running).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@MatteoMer @claude
Add MSM and pairing fixture-backed tests (P0)
2203f7a 
Completes the P0 fixture-backed test coverage by adding arkworks-validated
vector tests for MSM (G1 Fr and G1 i128) and pairing (generator scalar cases).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@MatteoMer @claude
Add shared microbenchmark harness, pairing benchmarks, and ARM64 dedi…
9dd8fcf 
…cated squaring

Shared bench harness (bench/zolt_arith/bench_harness.zig):
- Multi-sample collection (21 samples x configurable iterations)
- Statistical output: min/median/mean/p99/stddev with auto-scaled time units
- Structured [BENCH] output lines for grep/parse
- Config presets for field (~ns) and pairing (~us) timescales
- Zero external dependencies

Benchmarks:
- Refactored field_micro.zig to use the shared harness
- New pairing_micro.zig: pairingFp, millerLoop, millerLoopPrepared, finalExp
- New build step: zig build bench-zolt-arith-pairing

ARM64 Montgomery squaring (arm64MontgomerySquare256):
- Dedicated squaring routine exploiting a[i]*a[j] == a[j]*a[i] symmetry
- Product phase: 10 multiplies (6 cross + 4 diagonal) vs 16 for generic mul
- Reduction phase: 4 CIOS iterations with carry propagation into upper limbs
- ~5% speedup on Fp.square (17.1ns vs 18.1ns), all 158 tests passing
- Wired into both MontgomeryField.square and BN254Scalar.square

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@MatteoMer @claude
Add x86-64 dedicated Montgomery squaring (ADX)
0e0da8c 
Mirrors the ARM64 squaring optimization for x86-64 using mulxq/adcxq/adoxq
dual carry chains. Same product-then-reduce approach: 10 multiplies for a^2
(6 cross + 4 diagonal) instead of 16, then 4 CIOS reduction iterations with
carry propagation into upper product limbs.

Added montgomerySquareX86() to both MontgomeryField and BN254Scalar, wired
into their square() methods on the use_asm_mul path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@MatteoMer @claude
Add extension field (Fp2/Fp6/Fp12) differential fixtures (P1)
9a9e8ba 
Adds arkworks-based fixture generation and Zig differential verification
for the full BN254 extension field tower. Covers add, sub, mul, square,
inverse, conjugate, frobenius (1/2/3), and cyclotomic square across
~165 test vectors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@MatteoMer @claude
Add remaining P1 differential fixtures (G2, compression, GLV, G2 MSM,…
bcc3e93 
… Dory, GPU)

Extend the arkworks fixture generator with 5 new categories and add
corresponding differential + package-level tests for all remaining P1
items from the test matrix:

- G2 curve ops: scalar_mul, add, double, neg (16 vectors)
- Point compression: G1/G2 compress roundtrip (8 vectors each)
- GLV: G1/G2 scalar mul via arkworks oracle (10 vectors each)
- G2 MSM: extend sizes from [1,2,4,8] to [1,2,4,8,16,32]
- Dory commitment: replicate Zolt SRS hash-to-curve in Rust (nv=2,3,4)
- GPU field crossover: CPU-computed expected values at sizes 4,16,64

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@MatteoMer @claude
fix: correct fp2Sqrt fallback that broke G2 decompression
9eb8d3f 
The fallback branch in fp2Sqrt negated the candidate before calling
tonelliShanks, converting a guaranteed QR into a guaranteed non-QR
(since p ≡ 3 mod 4 for BN254, negation flips quadratic residuosity).
This caused decompressG2 to return null for every non-identity point.

The fix unifies both branches: try c² = (a+t)/2, then c² = (a-t)/2,
and derive d = b/(2c) in both cases. G2 compression tests now verify
full compress→decompress roundtrip including y-coordinates.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@MatteoMer @claude
Add CI jobs for differential fixtures, freshness check, and benchmarks
a1f37d4 
- zolt-arith-diff: runs all 21 differential test blocks (blocking)
- zolt-arith-fixture-freshness: regenerates fixtures and fails on drift (blocking)
- zolt-arith-bench: field + pairing microbench with [BENCH] summary (non-blocking)
- Package-local bench-field and bench-pairing forwarding steps

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@MatteoMer MatteoMer force-pushed the zolt-arith-fixtures-research branch from 057b2f0 to a1f37d4 Compare April 5, 2026 19:31
@MatteoMer MatteoMer merged commit 8a32f3a into main Apr 5, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@MatteoMer MatteoMer

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@MatteoMer