Mitigate proof-deserialization DOS vector

[moodlezoup]

Summary

Adds a canonical, size-bounded decode boundary for JoltProof in jolt-verifier, closing a deserialization DoS vector: a standalone proof arrives as untrusted bytes, and its many length-prefixed Vecs (RA commitment vectors, per-round sumcheck vectors, BlindFold row/commitment vectors) previously had no systematic allocation ceiling — a malicious length prefix could drive an unbounded allocation. (The Dory round count was already capped element-locally by MAX_SERIALIZED_PROOF_ROUNDS; this is the outer bound covering everything else.)

• JoltProof::from_canonical_bytes / from_canonical_bytes_bounded (crates/jolt-verifier/src/decode.rs): rejects inputs over MAX_PROOF_BYTES (128 MiB) before allocating, caps allocation during decoding via bincode's read limit, and rejects trailing bytes after a complete proof so exactly one encoding is accepted (no proof malleability). Callers with a tighter bound from preprocessing can pass it; MAX_PROOF_BYTES remains the absolute ceiling.
• JoltProof::to_canonical_bytes: the matching encoder.
• New VerifierError variants: ProofTooLarge, TrailingProofBytes, ProofDeserializationFailed, ProofSerializationFailed.

Element-level validation (curve points in-subgroup, field elements reduced) is unchanged — it's enforced by the field/group serde impls routing through CanonicalDeserialize with validation.

Testing

• Unit tests on the verifier-model proof: round-trip, trailing-byte rejection, size-cap rejection (crates/jolt-verifier/src/verifier.rs).
• core-fixtures integration test round-trips a real Dory muldiv proof through the canonical encoding and checks size-cap, trailing-byte, and truncation rejections end to end (crates/jolt-verifier/tests/support/core_fixtures.rs).
• Compile-time assertion that the production (DoryScheme, Pedersen<Bn254G1>) instantiation satisfies the required serde bounds.

🤖 Generated with Claude Code

[github-actions]

Benchmark comparison (crates)

group                           main_run                               pr_run
-----                           --------                               ------
EqPolynomial::evals/17          1.05      2.1±0.02ms        ? ?/sec    1.00  1958.8±79.29µs        ? ?/sec
EqPolynomial::evals/19          1.10      8.0±0.06ms        ? ?/sec    1.00      7.2±0.06ms        ? ?/sec
EqPolynomial::evals/20          1.10     16.4±0.10ms        ? ?/sec    1.00     14.9±0.09ms        ? ?/sec
EqPolynomial::evals/22          1.09     62.8±0.48ms        ? ?/sec    1.00     57.5±0.40ms        ? ?/sec
EqPolynomial::evaluations/14    1.00   621.8±19.41µs        ? ?/sec    1.27   790.6±15.81µs        ? ?/sec
EqPolynomial::evaluations/18    1.13      7.2±0.09ms        ? ?/sec    1.00      6.4±0.05ms        ? ?/sec
EqPolynomial::evaluations/20    1.10     30.7±0.21ms        ? ?/sec    1.00     27.9±0.96ms        ? ?/sec
Fr::mul_u128                    1.10     24.7±0.56ns        ? ?/sec    1.00     22.4±0.25ns        ? ?/sec
Fr::mul_u64                     1.07     12.3±0.11ns        ? ?/sec    1.00     11.5±0.11ns        ? ?/sec
Fr::to_bytes                    1.15     20.1±0.26ns        ? ?/sec    1.00     17.5±0.17ns        ? ?/sec
Polynomial::bind/14             1.18    153.9±2.91µs        ? ?/sec    1.00    130.0±4.14µs        ? ?/sec
Polynomial::bind/18             1.12      2.2±0.03ms        ? ?/sec    1.00  1939.6±24.84µs        ? ?/sec
Polynomial::bind/20             1.10      8.5±0.06ms        ? ?/sec    1.00      7.7±0.16ms        ? ?/sec
Polynomial::evaluate/20         1.09     44.6±1.51ms        ? ?/sec    1.00     40.9±1.04ms        ? ?/sec
append_bytes/Blake2b/256B       1.05   345.8±11.90ns        ? ?/sec    1.00    329.3±2.48ns        ? ?/sec
append_bytes/Keccak/256B        1.12   924.5±13.74ns        ? ?/sec    1.00    824.4±4.38ns        ? ?/sec
append_bytes/Poseidon/32B       1.06     97.9±1.80µs        ? ?/sec    1.00     92.7±0.90µs        ? ?/sec
challenge/Blake2b               1.14   727.7±19.87ns        ? ?/sec    1.00    638.1±8.15ns        ? ?/sec
challenge/Keccak                1.12   441.3±12.44ns        ? ?/sec    1.00    394.9±1.56ns        ? ?/sec
challenge/Poseidon              1.06     48.9±0.48µs        ? ?/sec    1.00     46.2±0.71µs        ? ?/sec
g1_add                          1.15   470.0±23.09ns        ? ?/sec    1.00   409.2±10.99ns        ? ?/sec
g1_double                       1.11    235.1±4.45ns        ? ?/sec    1.00    212.5±6.80ns        ? ?/sec
g1_msm/1024                     1.10     14.1±0.04ms        ? ?/sec    1.00     12.8±0.07ms        ? ?/sec
g1_msm/16                       1.13   603.7±12.63µs        ? ?/sec    1.00    534.7±4.58µs        ? ?/sec
g1_msm/256                      1.11      4.7±0.09ms        ? ?/sec    1.00      4.3±0.02ms        ? ?/sec
g1_msm/4                        1.12    286.4±1.08µs        ? ?/sec    1.00   256.8±11.97µs        ? ?/sec
g1_scalar_mul                   1.12     72.3±1.67µs        ? ?/sec    1.00     64.6±0.89µs        ? ?/sec
g1_serialize_bincode            1.08    119.9±1.46ns        ? ?/sec    1.00    110.6±0.62ns        ? ?/sec
g2_msm/256                      1.12     17.5±0.03ms        ? ?/sec    1.00     15.6±0.04ms        ? ?/sec
g2_msm/4                        1.11    867.3±9.74µs        ? ?/sec    1.00    778.3±4.35µs        ? ?/sec
g2_msm/64                       1.13      6.4±0.01ms        ? ?/sec    1.00      5.6±0.02ms        ? ?/sec
g2_scalar_mul                   1.12   362.0±10.26µs        ? ?/sec    1.00    323.0±2.64µs        ? ?/sec
gt_scalar_mul                   1.00   782.5±21.74µs        ? ?/sec    1.14    892.2±6.05µs        ? ?/sec
multi_pairing/16                1.10      6.4±0.02ms        ? ?/sec    1.00      5.8±0.02ms        ? ?/sec
multi_pairing/2                 1.00   1067.7±8.37µs        ? ?/sec    1.18   1255.7±6.57µs        ? ?/sec
multi_pairing/4                 1.07  1970.8±13.66µs        ? ?/sec    1.00  1849.5±11.14µs        ? ?/sec
multi_pairing/8                 1.10      3.5±0.01ms        ? ?/sec    1.00      3.2±0.04ms        ? ?/sec
pairing                         1.00    803.8±7.07µs        ? ?/sec    1.16   932.9±30.98µs        ? ?/sec
pedersen_commit/1024            1.10     14.1±0.02ms        ? ?/sec    1.00     12.9±0.02ms        ? ?/sec
pedersen_commit/16              1.13   679.8±17.31µs        ? ?/sec    1.00    602.9±6.42µs        ? ?/sec
pedersen_commit/256             1.11      4.8±0.10ms        ? ?/sec    1.00      4.4±0.01ms        ? ?/sec
pedersen_commit/4               1.12    357.8±3.52µs        ? ?/sec    1.00    318.4±3.15µs        ? ?/sec