feat(chacha12): Consolidate PRNG for fuzzing#3232
Conversation
🦋 Changeset detectedLatest commit: cc336d4 The changes in this PR will be included in the next version bump. This PR includes changesets to release 5 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
gibson042
left a comment
gibson042
left a comment
There was a problem hiding this comment.
Some thoughts and suggestions on design and implementation.
I'd really like to see a more isolated core that can be used to construct multiple interfaces such as proposal-random-functions Random and pure-rand RandomGenerator.
| the extra speed is generally the right tradeoff. | ||
|
|
||
| The keystream is verified against three published test vectors | ||
| from `draft-strombergson-chacha-test-vectors` (TC1, TC4, TC8). |
There was a problem hiding this comment.
Might as well make this a hyperlink to https://datatracker.ietf.org/doc/html/draft-strombergson-chacha-test-vectors-01 .
| | Field | Value | | ||
| | -------- | ---------------------------------------------- | | ||
| | CPU | AMD Ryzen AI MAX+ 395 w/ Radeon 8060S (32 vCPU) | | ||
| | RAM | 128 GiB | | ||
| | OS | Linux 6.14 (Ubuntu 24.04) | | ||
| | Node | 22.22.2 | | ||
| | Arch | x64 | | ||
| | Harness | `packages/chacha12/test/random.bench.js` | |
| This package is the ChaCha12 sibling of `@endo/chacha20` (the | ||
| 20-round variant) and `@endo/xorshift` (a non-cryptographic PRNG); | ||
| choose among them by package name based on the throughput vs margin | ||
| tradeoff a given consumer needs. |
There was a problem hiding this comment.
Do all three packages cross-reference the other two? It seems valuable for them to implement and maintain a consistent API, and I'm wondering whether that and other commonalities should rely upon something like importing from an @endo/prng package.
There was a problem hiding this comment.
Hmm, the other two packages don't even exist! Comment stands if there are plans to introduce them, but otherwise:
| This package is the ChaCha12 sibling of `@endo/chacha20` (the | |
| 20-round variant) and `@endo/xorshift` (a non-cryptographic PRNG); | |
| choose among them by package name based on the throughput vs margin | |
| tradeoff a given consumer needs. |
| The PRNG method names align with the | ||
| [TC39 proposal-random-functions](https://tc39.es/proposal-random-functions/) | ||
| (Stage 1) so consumers can swap in a `Random.Seeded` instance later | ||
| without changing call sites. |
There was a problem hiding this comment.
Given that the proposal is at stage 1, this claim is too strong. I'd prefer to pare down the API to the point where such a swap becomes possible in the future without breaking changes here regardless of dramatic evolution in the proposal. Even if we need all four abilities (draw sub-unitary non-negative float, draw integer from closed interval, draw fresh Uint8Array, fill Uint8Array), perhaps it makes sense to start with just a block stream here and a more ergonomic translating wrapper in the previously-suggested common package?
import { BLOCK_SIZE, makeChaCha12 } from '@endo/chacha12';
import { decodeHex } from '@endo/hex';
import { makeRandomFromBlockFiller } from '@endo/prng';
const { pullBlock }: { pullBlock: (out: Uint8Array) => void } = makeChaCha12(
decodeHex('b0b5c0ffeefacade'.repeat(4)),
);
// `makeRandomFromBlockFiller` makes a proposal-random-functions Random from a
// `pullBlock`-shaped Uint8Array-filling function.
const prng: {
random: () => number;
int: (lo: number, hi: number) => number;
bytes: (n: number) => Uint8Array;
fillBytes: (buf: Uint8Array, start?: number, end?: number) => Uint8Array;
} = makeRandomFromBlockFiller(pullBlock, { blockSize: BLOCK_SIZE });
assert(0 <= prng.random() && prng.random() < 1);
assert(Math.abs(prng.int(10, 20) - 15) <= 5);There was a problem hiding this comment.
And for bonus points, we should also be able to represent a ChaCha12 as a pure-rand RandomGenerator for compatibility with fast-check randomType.
import fc from 'fast-check';
import type { RandomType as FastCheckRandomType } from 'fast-check';
import type { RandomGenerator } from 'pure-rand/types/RandomGenerator';
import {
BLOCK_SIZE,
makeChaCha12Kit,
adaptToPureRandomGenerator,
} from '@endo/chacha12';
/*
const adaptToPureRandomGenerator = ({
getState,
pullBlock,
}: ChaCha12Kit): RandomGenerator => {
const buf8 = new Uint8Array(BLOCK_SIZE);
const buf32 = new Int32Array(buf8.buffer);
let prevBuf32Idx = 0;
return {
clone: () =>
adaptToPureRandomGenerator(makeChaCha12Kit({ state: getState() })),
next: () => {
if (prevBuf32Idx === 0) {
// Refill and reset.
pullBlock(buf8);
prevBuf32Idx = buf32.length;
}
prevBuf32Idx -= 1;
return buf32[prevBuf32Idx];
},
getState: () => getState(),
};
};
*/
const randomGenerator: RandomGenerator = adaptToPureRandomGenerator(
makeChaCha12Kit({ seed: decodeHex('b0b5c0ffeefacade'.repeat(4)) }),
);
// Our fast-check adapter.
const chaCha12RandomType: FastCheckRandomType = (int32Seed: number) => {
const chaChaSeed = new Int32Array(8).fill(int32Seed);
return adaptToPureRandomGenerator(
makeChaCha12Kit({ seed: new Uint8Array(chaChaSeed.buffer) }),
);
};
// A deterministically failing PBT fed by ChaCha12.
fc.assert(
fc.property(fc.integer(), n => n >= 0),
{ randomType: chaCha12RandomType, seed: 0xdeadbeef },
);| - `random()` reads 8 keystream bytes, masks the top 11 bits, and | ||
| divides by `2 ** 53`. | ||
| Each respective returned value from streams with the same seed is | ||
| equal across runs and engines because the float is computed by | ||
| dividing a 53-bit integer by an exact power of two, with no | ||
| engine-dependent rounding. |
There was a problem hiding this comment.
| - `random()` reads 8 keystream bytes, masks the top 11 bits, and | |
| divides by `2 ** 53`. | |
| Each respective returned value from streams with the same seed is | |
| equal across runs and engines because the float is computed by | |
| dividing a 53-bit integer by an exact power of two, with no | |
| engine-dependent rounding. | |
| - `random()` ensures that each respective returned value from streams with the | |
| same seed is equal across runs and engines by internally constructing a 53-bit | |
| integer and dividing it by `2 ** 53` (which avoids engine-dependent rounding). |
| /** | ||
| * Pulls 8 keystream bytes, masks the high 11 bits to 0, and | ||
| * divides the resulting 53-bit integer by `2 ** 53`. The result | ||
| * is a float in `[0, 1)`, deterministic for a given seed across | ||
| * engines. | ||
| */ |
There was a problem hiding this comment.
| /** | |
| * Pulls 8 keystream bytes, masks the high 11 bits to 0, and | |
| * divides the resulting 53-bit integer by `2 ** 53`. The result | |
| * is a float in `[0, 1)`, deterministic for a given seed across | |
| * engines. | |
| */ | |
| /** | |
| * Produces a float in `[0, 1)` that is deterministic for a given seed | |
| * across all engines by constructing an unsigned 53-bit integer and | |
| * dividing it by `2 ** 53`. | |
| */ |
| // Read a 32-bit unsigned little-endian integer from the keystream. | ||
| // Used by `int()` for ranges that fit in 32 bits, avoiding the | ||
| // float round-trip through `random()`. | ||
| const readU32 = () => { | ||
| const b0 = readByte(); | ||
| const b1 = readByte(); | ||
| const b2 = readByte(); | ||
| const b3 = readByte(); | ||
| return (b0 | (b1 << 8) | (b2 << 16) | (b3 << 24)) >>> 0; | ||
| }; |
There was a problem hiding this comment.
| // Read a 32-bit unsigned little-endian integer from the keystream. | |
| // Used by `int()` for ranges that fit in 32 bits, avoiding the | |
| // float round-trip through `random()`. | |
| const readU32 = () => { | |
| const b0 = readByte(); | |
| const b1 = readByte(); | |
| const b2 = readByte(); | |
| const b3 = readByte(); | |
| return (b0 | (b1 << 8) | (b2 << 16) | (b3 << 24)) >>> 0; | |
| }; | |
| /** Read a 16-bit unsigned little-endian integer from the keystream. */ | |
| const readU16 = () => { | |
| const b0 = readByte(); | |
| const b1 = readByte(); | |
| return (b0 | (b1 << 8)) >>> 0; | |
| }; | |
| /** Read a 24-bit unsigned little-endian integer from the keystream. */ | |
| const readU24 = () => { | |
| const b0 = readByte(); | |
| const b1 = readByte(); | |
| const b2 = readByte(); | |
| return (b0 | (b1 << 8) | (b2 << 16)) >>> 0; | |
| }; | |
| /** Read a 32-bit unsigned little-endian integer from the keystream. */ | |
| const readU32 = () => { | |
| const b0 = readByte(); | |
| const b1 = readByte(); | |
| const b2 = readByte(); | |
| const b3 = readByte(); | |
| return (b0 | (b1 << 8) | (b2 << 16) | (b3 << 24)) >>> 0; | |
| }; | |
| /** Read a 53-bit unsigned little-endian integer from the keystream. */ | |
| const readU53 = () => { | |
| // Assemble the integer from two 32-bit halves, masking off the top | |
| // 11 bits of the upper half to leave 21 high bits + 32 low bits = | |
| // 53 bits. | |
| const lo = readU32(); | |
| const hi24 = readU24(); | |
| const hi21 = hi24 & 0x1fffff; | |
| return hi21 * 4294967296 + lo; | |
| }; |
| const random = () => { | ||
| // Read 8 bytes from the buffer; assemble into a non-negative | ||
| // 53-bit integer using two halves. | ||
| // | ||
| // High 32 bits: little-endian. We mask off the top 11 bits to | ||
| // leave 21 high bits + 32 low bits = 53 bits. | ||
| const b0 = readByte(); | ||
| const b1 = readByte(); | ||
| const b2 = readByte(); | ||
| const b3 = readByte(); | ||
| const b4 = readByte(); | ||
| const b5 = readByte(); | ||
| const b6 = readByte(); | ||
| const b7 = readByte(); | ||
| const lo = (b0 | (b1 << 8) | (b2 << 16) | (b3 << 24)) >>> 0; | ||
| // Mask top 11 bits of the high word to 0, keeping 21 bits. | ||
| const hi = ((b4 | (b5 << 8) | (b6 << 16) | (b7 << 24)) >>> 0) & 0x1fffff; | ||
| // hi * 2 ** 32 + lo, divided by 2 ** 53: | ||
| // = hi * 2 ** -21 + lo * 2 ** -53 | ||
| // Computed as a single multiply by POW2_M53 of the 53-bit int. | ||
| return (hi * 4294967296 + lo) * POW2_M53; | ||
| }; |
There was a problem hiding this comment.
| const random = () => { | |
| // Read 8 bytes from the buffer; assemble into a non-negative | |
| // 53-bit integer using two halves. | |
| // | |
| // High 32 bits: little-endian. We mask off the top 11 bits to | |
| // leave 21 high bits + 32 low bits = 53 bits. | |
| const b0 = readByte(); | |
| const b1 = readByte(); | |
| const b2 = readByte(); | |
| const b3 = readByte(); | |
| const b4 = readByte(); | |
| const b5 = readByte(); | |
| const b6 = readByte(); | |
| const b7 = readByte(); | |
| const lo = (b0 | (b1 << 8) | (b2 << 16) | (b3 << 24)) >>> 0; | |
| // Mask top 11 bits of the high word to 0, keeping 21 bits. | |
| const hi = ((b4 | (b5 << 8) | (b6 << 16) | (b7 << 24)) >>> 0) & 0x1fffff; | |
| // hi * 2 ** 32 + lo, divided by 2 ** 53: | |
| // = hi * 2 ** -21 + lo * 2 ** -53 | |
| // Computed as a single multiply by POW2_M53 of the 53-bit int. | |
| return (hi * 4294967296 + lo) * POW2_M53; | |
| }; | |
| const random = () => readU53() * POW2_M53; |
| // Fast path: ranges up to `2 ** 32` use a single 32-bit | ||
| // keystream draw and pure integer arithmetic to eliminate modulo | ||
| // bias. Discard draws in `[limit32, 2 ** 32)` so the retained | ||
| // `limit32` buckets divide evenly into `range`. | ||
| if (range <= 0x100000000) { | ||
| const limit32 = 0x100000000 - (0x100000000 % range); | ||
| for (;;) { | ||
| const u = readU32(); | ||
| if (u < limit32) { | ||
| return lo + (u % range); | ||
| } | ||
| } | ||
| } | ||
| // Slow path: ranges up to `2 ** 53`. Rejection sampling on a | ||
| // 53-bit integer drawn directly from the keystream as a pair of | ||
| // 32-bit halves; the 11 high bits of the high word are masked | ||
| // off. No floating-point arithmetic is involved in selecting | ||
| // the integer. | ||
| // | ||
| // `limit53 = floor(2 ** 53 / range) * range`. Discard draws in | ||
| // `[limit53, 2 ** 53)` and return `lo + (u % range)`. | ||
| const limit53 = Math.floor(9007199254740992 / range) * range; | ||
| for (;;) { | ||
| const lo32 = readU32(); | ||
| const hi21 = readU32() & 0x1fffff; | ||
| const u = hi21 * 4294967296 + lo32; // exact 53-bit non-negative int. | ||
| if (u < limit53) { | ||
| return lo + (u % range); | ||
| } | ||
| } |
There was a problem hiding this comment.
Let's eliminate some pathologies (e.g., int(0, 2**31) should not throw away almost half of the draws because its upper acceptable limit ends up at 2**31 + 1).
| // Fast path: ranges up to `2 ** 32` use a single 32-bit | |
| // keystream draw and pure integer arithmetic to eliminate modulo | |
| // bias. Discard draws in `[limit32, 2 ** 32)` so the retained | |
| // `limit32` buckets divide evenly into `range`. | |
| if (range <= 0x100000000) { | |
| const limit32 = 0x100000000 - (0x100000000 % range); | |
| for (;;) { | |
| const u = readU32(); | |
| if (u < limit32) { | |
| return lo + (u % range); | |
| } | |
| } | |
| } | |
| // Slow path: ranges up to `2 ** 53`. Rejection sampling on a | |
| // 53-bit integer drawn directly from the keystream as a pair of | |
| // 32-bit halves; the 11 high bits of the high word are masked | |
| // off. No floating-point arithmetic is involved in selecting | |
| // the integer. | |
| // | |
| // `limit53 = floor(2 ** 53 / range) * range`. Discard draws in | |
| // `[limit53, 2 ** 53)` and return `lo + (u % range)`. | |
| const limit53 = Math.floor(9007199254740992 / range) * range; | |
| for (;;) { | |
| const lo32 = readU32(); | |
| const hi21 = readU32() & 0x1fffff; | |
| const u = hi21 * 4294967296 + lo32; // exact 53-bit non-negative int. | |
| if (u < limit53) { | |
| return lo + (u % range); | |
| } | |
| } | |
| // Draw the fewest bytes necessary to either match the requested range or | |
| // cover at least two repetitions of it, discarding draws in the range | |
| // beyond the last full repetition to avoid modulo bias while keeping the | |
| // discard fraction acceptably low. | |
| const { draw, limit } = (() => { | |
| // Read one byte to cover a range of exactly 256 or up to 128. | |
| if (range === 0x100 || range <= 0x80) { | |
| return { draw: readByte, limit: 0x100 - (0x100 % range) }; | |
| } | |
| // Read two bytes to cover a range of exactly 65536 or up to 32768. | |
| if (range === 0x10000 || range <= 0x8000) { | |
| return { draw: readU16, limit: 0x10000 - (0x10000 % range) }; | |
| } | |
| // ...and so on. | |
| if (range === 0x1000000 || range <= 0x800000) { | |
| return { draw: readU24, limit: 0x1000000 - (0x1000000 % range) }; | |
| } | |
| if (range === 0x100000000 || range <= 0x80000000) { | |
| return { draw: readU32, limit: 0x100000000 - (0x100000000 % range) }; | |
| } | |
| // The final limit is a 53-bit integer, which will discard just under half | |
| // of all possible draws when the range is just slightly above 2**52. | |
| return { | |
| draw: readU53, | |
| limit: Math.floor(9007199254740992 / range) * range, | |
| }; | |
| })(); | |
| for (;;) { | |
| const n = draw(); | |
| if (n < limit) { | |
| return lo + (n % range); | |
| } | |
| } |
| if (!(seed instanceof Uint8Array)) { | ||
| throw TypeError('seed must be a Uint8Array'); | ||
| } | ||
| if (seed.length !== 32) { | ||
| throw TypeError( | ||
| `seed must be 32 bytes (got ${seed.length}); ChaCha12 keys are 256 bits`, | ||
| ); | ||
| } |
There was a problem hiding this comment.
Suggestion: align with patterns in chacha12.js.
| if (!(seed instanceof Uint8Array)) { | |
| throw TypeError('seed must be a Uint8Array'); | |
| } | |
| if (seed.length !== 32) { | |
| throw TypeError( | |
| `seed must be 32 bytes (got ${seed.length}); ChaCha12 keys are 256 bits`, | |
| ); | |
| } | |
| if (!(seed instanceof Uint8Array) || seed.length !== 32) { | |
| throw TypeError('ChaCha12 seed must be a 32-byte Uint8Array'); | |
| } |
Per Richard Gibson's review of #3232: pare @endo/chacha12 to the keystream primitive and introduce @endo/random as a source-agnostic library of per-function samplers. @endo/random exposes randomInt, randomBytes, randomFillBytes, and random as separate tree-shakeable modules, each parameterized by a RandomSource (an object with a single fillBytes method). Names follow the TC39 proposal-random-functions translation Random.method -> randomMethod. Adapters in tools/ bridge to a fixed-block-size keystream filler, to/from pure-rand's RandomGenerator (the contract fast-check uses), and a fast-check randomType helper. tools/test-seeds publishes the canonical Bob's Coffee Façade seed. @endo/chacha12 now exports BLOCK_SIZE, chacha12Block, chacha12State (with optional nonce/counter), makeChaCha12Source, and assertChaCha12Key only. The sampling functions and their tests have moved to @endo/random; chacha12 keystream KAT vectors stay. Adopted Richard's specific suggestions: range-aware rejection sampling in randomInt (single-byte path for ranges <= 128 etc.), readU16/24/32 /53 helpers, random = readU53 * 2^-53, chacha12State(key) shorthand, hyperlink to draft-strombergson-chacha-test-vectors-01, BENCH.md table alignment, and a shared seed file. Strict-fork clone() semantics on RandomSource are deferred. Rewires @endo/hex benches and @endo/ocapn fuzz tests to draw bytes from @endo/chacha12 through @endo/random samplers using the shared seed. All 24 random tests, 9 chacha12 keystream tests, 13 hex tests, and 282 ocapn tests pass under both lockdown and shims-only configs.
kriskowal
force-pushed
the
kriskowal-random-chacha20
branch
from
7fd7a67 to
9456fb1
Compare
|
@gibson042 — thank you for the thorough first-round review. The PR has gone through several redesign rounds since (PR #75 in the mirror tracks the work-in-progress); each of your comments is now addressed. Mapping comment-by-comment: Surface redesigncomment 3177082445 ( comment 3177216582 (pure-rand comments 3176918766 / 3177404457 (cross-reference among packages, "the other two packages don't even exist"). Resolved per your second comment: speculative references to chacha12 keystreamcomment 3177309309 ( comment 3177312921 ( comment 3177399312 (
|
Per Richard Gibson's review of #3232: pare @endo/chacha12 to the keystream primitive and introduce @endo/random as a source-agnostic library of per-function samplers. @endo/random exposes randomInt, randomBytes, randomFillBytes, and random as separate tree-shakeable modules, each parameterized by a RandomSource (an object with a single fillBytes method). Names follow the TC39 proposal-random-functions translation Random.method -> randomMethod. Adapters in tools/ bridge to a fixed-block-size keystream filler, to/from pure-rand's RandomGenerator (the contract fast-check uses), and a fast-check randomType helper. tools/test-seeds publishes the canonical Bob's Coffee Façade seed. @endo/chacha12 now exports BLOCK_SIZE, chacha12Block, chacha12State (with optional nonce/counter), makeChaCha12Source, and assertChaCha12Key only. The sampling functions and their tests have moved to @endo/random; chacha12 keystream KAT vectors stay. Adopted Richard's specific suggestions: range-aware rejection sampling in randomInt (single-byte path for ranges <= 128 etc.), readU16/24/32 /53 helpers, random = readU53 * 2^-53, chacha12State(key) shorthand, hyperlink to draft-strombergson-chacha-test-vectors-01, BENCH.md table alignment, and a shared seed file. Strict-fork clone() semantics on RandomSource are deferred. Rewires @endo/hex benches and @endo/ocapn fuzz tests to draw bytes from @endo/chacha12 through @endo/random samplers using the shared seed. All 24 random tests, 9 chacha12 keystream tests, 13 hex tests, and 282 ocapn tests pass under both lockdown and shims-only configs.
kriskowal
force-pushed
the
kriskowal-random-chacha20
branch
from
9456fb1 to
f40e94a
Compare
gibson042
left a comment
gibson042
left a comment
There was a problem hiding this comment.
The packages/random/$noun.js files that just export { … } from './src/$noun.js'; clutter up the package root; I understand why we don't want to expose the src/ files directly, but perhaps we should establish a convention of storing such files in an exports/ sibling directory.
Aside from that, I think the pure-rand v8 RandomSource interface would be a much better interop target than the v5 one.
Neither of these are blocking concerns, and the rest of the comments won't need another round of review.
| /** | ||
| * The shape of a `pure-rand` v5 `RandomGenerator`, the contract used | ||
| * by `fast-check` to drive property-based tests via the `randomType` | ||
| * parameter. We restate it locally rather than depend on `pure-rand` | ||
| * directly so `@endo/random` stays standalone. | ||
| * | ||
| * `next()` returns a `[value, nextGenerator]` tuple where `value` is | ||
| * a 32-bit signed integer in `[min(), max()]` (typically | ||
| * `[-0x80000000, 0x7fffffff]`). The returned generator MAY be the | ||
| * same instance with mutated state. | ||
| * | ||
| * `clone()` is required by `pure-rand` v5; for serial keystreams | ||
| * without state-snapshot support the adapter returns an alias whose | ||
| * state is shared with the original. | ||
| */ | ||
| export interface PureRandomGenerator { | ||
| next(): [number, PureRandomGenerator]; | ||
| unsafeNext(): number; | ||
| clone(): PureRandomGenerator; | ||
| min(): number; | ||
| max(): number; | ||
| } |
There was a problem hiding this comment.
Why pure-rand v5? The v8 interface is much better, and even preferred by fast-check: https://github.com/dubzzz/fast-check/blob/0858ecd84595c1245b8d26ca86e797643c2ad4e4/packages/fast-check/src/random/generator/RandomGenerator.ts
| /** | ||
| * Builds a `fast-check`-compatible `randomType` function from a | ||
| * factory that turns a 32-byte seed into a `RandomSource`. | ||
| * `fast-check` calls the returned function with a 32-bit signed | ||
| * integer seed and expects back a `pure-rand`-shaped | ||
| * `RandomGenerator`. The integer seed is broadcast across a 32-byte | ||
| * buffer (compatible with ChaCha-style key inputs). | ||
| * | ||
| * @param {(seed: Uint8Array) => RandomSource} makeSourceFromSeed | ||
| * @returns {(int32Seed: number) => PureRandomGenerator} | ||
| */ |
There was a problem hiding this comment.
This should probably link to https://fast-check.dev/docs/api/interfaces/Parameters/#randomtype .
| // Range-aware rejection sampling: each tier of the draw-width | ||
| // staircase covers a specific range domain. This test exercises the | ||
| // boundaries: single-byte (range <= 128 and exact 256), two-byte (up | ||
| // to 32768 and exact 65536), three-byte (up to 8388608), four-byte | ||
| // (up to 2^31 and exact 2^32), and the 53-bit slow path (range > | ||
| // 2^32). Each branch produces values in the requested closed | ||
| // interval. |
There was a problem hiding this comment.
Do we also want testing that verifies the expected count of read bytes for ranges of particular size?
| test('chaCha12-backed randomType drives fc.assert successfully', t => { | ||
| // A property that is always true exercises the adapter: fast-check | ||
| // invokes `chaCha12RandomType(seed)`, pulls values via | ||
| // RandomGenerator.next() / unsafeNext(), and reports no failures. | ||
| fc.assert( | ||
| fc.property(fc.integer(), () => true), | ||
| /** @type {any} */ ({ | ||
| randomType: chaCha12RandomType, | ||
| seed: 0xdeadbeef, | ||
| numRuns: 100, | ||
| }), | ||
| ); | ||
| t.pass(); | ||
| }); |
There was a problem hiding this comment.
Likewise here; can we extend this to verify that the value passed as randomType is called with the provided seed and that bytes are actually read from the value it returns?
Per Richard Gibson's review of endojs/endo#3232: pare @endo/chacha12 to the keystream primitive and introduce @endo/random as a source-agnostic library of per-function samplers. @endo/random exposes randomInt, randomBytes, randomFillBytes, and random as separate tree-shakeable modules, each parameterized by a RandomSource (an object with a single fillBytes method). Names follow the TC39 proposal-random-functions translation Random.method -> randomMethod. Adapters in tools/ bridge to a fixed-block-size keystream filler, to/from pure-rand's RandomGenerator (the contract fast-check uses), and a fast-check randomType helper. tools/test-seeds publishes the canonical Bob's Coffee Façade seed. @endo/chacha12 now exports BLOCK_SIZE, chacha12Block, chacha12State (with optional nonce/counter), makeChaCha12Source, and assertChaCha12Key only. The sampling functions and their tests have moved to @endo/random; chacha12 keystream KAT vectors stay. Adopted Richard's specific suggestions: range-aware rejection sampling in randomInt (single-byte path for ranges <= 128 etc.), readU16/24/32 /53 helpers, random = readU53 * 2^-53, chacha12State(key) shorthand, hyperlink to draft-strombergson-chacha-test-vectors-01, BENCH.md table alignment, and a shared seed file. Strict-fork clone() semantics on RandomSource are deferred. Rewires @endo/hex benches and @endo/ocapn fuzz tests to draw bytes from @endo/chacha12 through @endo/random samplers using the shared seed. All 24 random tests, 9 chacha12 keystream tests, 13 hex tests, and 282 ocapn tests pass under both lockdown and shims-only configs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kriskowal
force-pushed
the
kriskowal-random-chacha20
branch
2 times, most recently
from
72e7809 to
2effced
Compare
There’s a deeper reason. Not all tools and older versions of Node.js don’t recognize the exports directive in package.json, so mirroring key to value is necessary as a fallback. We could just put these thunks in src/ otherwise. |
|
Another note: I pulled out the fastcheck adapters. The depth of that conversation led me to understand that there are at least two random number adapters we would want depending on the fastcheck version and that warranted a dedicated package or or packages if we sought to use our prng instead of one that ships with fastcheck. I also missed a couple notes in the review and will finish revising Monday. |
What software specifically are you concerned about here? The files in question are ECMAScript modules that use
I don't think we need to support anything other than the pure-rand v8 RandomSource interface, which is preferred by the latest version of fast-check and is the only one for which I have a use case (because I have seen Node.js crashes in #3053 that I suspect were ultimately caused by the PRNG internal to fast-check). And by "support" I mean that it must be possible to write source text as shown in #3232 (comment) : import fc from 'fast-check'; // ^4.6.0
import type { RandomType as FastCheckRandomType } from 'fast-check'; // ^4.6.0
import type {
RandomGenerator as PureRandGenerator,
} from 'pure-rand/types/RandomGenerator'; // ^8.4.0
import { $EXPORT as makeChaCha12Source } from '@endo/chacha12;
import { adaptToChaCha12Seed, adaptToPureRandV8Generator } from './adapters.js';
// Our fast-check adapter.
const chaCha12RandomType: FastCheckRandomType = (int32Seed: number) => {
const randomGenerator: PureRandGenerator = adaptToPureRandV8Generator(
makeChaCha12Source({ seed: adaptToChaCha12Seed(int32Seed) }),
);
return randomGenerator;
};
// A deterministically failing PBT fed by ChaCha12.
fc.assert(
fc.property(fc.integer(), n => n >= 0),
{ randomType: chaCha12RandomType, seed: 0xdeadbeef },
);And the ability to write and use those |
|
The last hold out for exports support is eslint. I prompted a report that reveals there may be a workaround by using a forked plugin. endojs/endo-but-for-bots#218 |
Address upstream review feedback on PR #3232 by exposing the
keystream's internal state for introspection and resumption.
`makeChaCha12(key)` now returns a `ChaCha12Generator` record
`{ next, getState, clone, fillRandomBytes }` instead of a bare
`(out: Uint8Array) => void` function. The new shape is structurally
compatible with `pure-rand` v8's `RandomGenerator` (the contract
`fast-check@4` consumes via its `randomType` parameter):
- `next()` returns a signed int32 in `[-0x80000000, 0x7fffffff]`
by reading 4 little-endian keystream bytes.
- `getState()` returns a 34-element `readonly number[]` snapshot
`[base0..base15, counter, offset, block0..block15]`.
- `clone()` returns a fully independent generator at the same
position (not an alias).
- `fillRandomBytes(out)` preserves the original byte-keystream
entry point and conforms to `@endo/random`'s `RandomSource`.
Add `makeChaCha12FromState(state)` for deterministic resumption
from a snapshot, completing the pure-rand-style paired
`makeXxx` / `makeXxxFromState` factory convention.
Update in-tree consumers (`@endo/random`, `@endo/hex`,
`@endo/ocapn`) to destructure `fillRandomBytes` from the returned
record. The migration is mechanical:
-const fillRandomBytes = makeChaCha12(seed);
+const { fillRandomBytes } = makeChaCha12(seed);
Add a new private package `@endo/chacha12-fast-check-test` that
drives `fast-check@4`'s property-based tester through the new
chacha12 surface as its `randomType` source. The package depends
on `fast-check@^4` directly; the production-shaped fast-check /
pure-rand adapters remain in the separately-designed
`@endo/random-fast-check` package.
The new chacha12 unit tests cover the int32 contract for `next`,
the round-trip property for `getState` /
`makeChaCha12FromState` (across every offset 0..64 to catch
block-boundary off-by-ones), `clone` independence, and malformed
state rejection. 18 chacha12 tests + 7 fast-check integration
tests pass.
Refs: endojs/endo#3232 (comment)
Refs: #75
|
|
||
| import { chacha12Block, chacha12State, makeChaCha12 } from '../src/chacha12.js'; | ||
|
|
||
| // Inline hex helpers: `@endo/hex` already depends on `@endo/chacha12` |
There was a problem hiding this comment.
Could break the cycle with a/the hex-test package.
… steward User correction after the liaison posted an explanatory comment on endojs/endo#3232 directly (kriskowal-identity) instead of routing through the steward (kriscendobot-identity). The principle: kriskowal's identity carries maintainer weight on primary repos (endojs/endo, agoric/agoric-sdk, etc.) and is reserved for actions that genuinely require it (reviews, approvals, merges). Comments are bot-side bookkeeping and belong to the bot. The post- handoff explanatory comment on an upstream PR after a boatman ferry should land via the steward, posted under kriscendobot. Mechanism: write a message-to-steward journal entry with the comment body and target PR; the steward picks it up on its next cycle and posts. Adds a small delay (one steward cycle) but preserves identity discipline. Touches: - roles/boatman/AGENT.md: new norm 'Comments on primary upstream repos route through the steward', alongside the existing 'Source- side cross-link only' norm. Pushes still go under kriskowal (gated by identity_switch_authorized); comments do not. - roles/liaison/AGENT.md: new operating norm 'No comments on primary repos under the kriskowal identity'. Carve-out: comments on the garden's own repos (endojs/endo-but-for-bots, kriskowal/garden) under kriskowal are fine. Those are the garden, not primary.
kriskowal
force-pushed
the
kriskowal-random-chacha20
branch
from
4d3c969 to
04664e5
Compare
| ### Why | ||
|
|
||
| [Upstream review feedback on PR #3232](https://github.com/endojs/endo/pull/3232#issuecomment-4421637048) | ||
| asked for `@endo/chacha12` to expose enough of its keystream state | ||
| to support a `pure-rand` v8 / `fast-check` v4 adapter. The closure | ||
| returned by the previous shape encapsulated `baseState`, `counter`, | ||
| `offset`, and `block` with no accessor, so an out-of-package | ||
| adapter could not snapshot or clone the keystream. | ||
|
|
||
| The new sibling package `@endo/chacha12-fast-check-test` (private) | ||
| exercises the surface end-to-end against `fast-check@4`'s | ||
| `randomType` parameter. |
There was a problem hiding this comment.
Please omit gratuitous process comment.
| - `@endo/random/fast-check.js` -- bidirectional adapters between | ||
| `RandomSource` and `pure-rand`'s `RandomGenerator` (the contract | ||
| `fast-check` uses via the `randomType` parameter), plus a | ||
| `randomType` builder. Imports nothing from `pure-rand` or | ||
| `fast-check`; depends only on those interfaces being structurally | ||
| compatible. |
| Add `@endo/chacha12`: a pure-JavaScript ChaCha12 keystream. The | ||
| factory `makeChaCha12(key)` returns a `(out: Uint8Array) => void` | ||
| function that conforms to `@endo/random`'s `RandomSource` and to | ||
| `crypto.getRandomValues`-style ergonomics; pass it directly to the | ||
| samplers. The keystream is cross-checked against three published | ||
| ChaCha12 test vectors from | ||
| [`draft-strombergson-chacha-test-vectors-01`](https://datatracker.ietf.org/doc/html/draft-strombergson-chacha-test-vectors-01) | ||
| (TC1, TC4, TC8). |
There was a problem hiding this comment.
The interface documented here is stale. We changed to an interface consistent with pure-rand-v8, structurally, with a fillRandomBytes function that satisfies RandomSource, like crypto.getRandomBytes.
| Rewires the `@endo/hex` encode/decode benchmarks and the | ||
| `@endo/ocapn` syrup and passable fuzz tests to draw deterministic | ||
| bytes from `@endo/chacha12` directly, using the shared seed from | ||
| `@endo/random/seeds.js`. | ||
|
|
There was a problem hiding this comment.
Omit as not interesting to package authors updating dependencies.
There was a problem hiding this comment.
We have only made one change in this release cycle, when this PR merges. Please consolidate these changeset.
| * @returns {ChaCha12Generator} | ||
| */ | ||
| const makeGenerator = ( | ||
| baseState, |
There was a problem hiding this comment.
Confirmed that we are indeed retaining this. That’s a foot-gun. Please allocate your own copy.
| // Engine-portable nanosecond timer. | ||
| const hasHrtime = | ||
| typeof globalThis.process === 'object' && | ||
| globalThis.process !== null && | ||
| typeof globalThis.process.hrtime === 'function' && | ||
| typeof globalThis.process.hrtime.bigint === 'function'; | ||
| const nowNs = hasHrtime | ||
| ? () => Number(globalThis.process.hrtime.bigint()) | ||
| : () => Date.now() * 1_000_000; |
There was a problem hiding this comment.
We can, frankly, presume that Ava tests are running under Node.js 20+.
| // from `block` to `out` we route through `set` after a subarray view. | ||
| // This is essentially the same as `copySet`; included as a sanity | ||
| // probe. | ||
| const copyCopywin = (out, i, block, offset, n) => { |
There was a problem hiding this comment.
I would call this copyWithin
There was a problem hiding this comment.
This needs to be reframed for this package and mostly deleted.
| | TC39 proposal | `@endo/random` | | ||
| | --- | --- | | ||
| | `Random.random()` | `random(source)` | | ||
| | `Random.int(lo, hi)` | `randomInt(source, lo, hi)` | |
There was a problem hiding this comment.
Use prettier on this. Leave a note to do this in general for all new or altered Markdown documents going forward.
Three recurring patterns from kriskowal's review of endojs/endo#3232: - Changeset hygiene (extend skills/changeset-discipline): one changeset per release cycle, keep the body current as the PR evolves, omit implementation detail, no process commentary. - Gratuitous renames (new skills/rename-discipline): leave existing identifiers alone unless the rename earns its place in the diff; a "gratuitous rename" review comment is a revert, not a defense. - Equivalence assertion (extend skills/regression-evidence): a code comment or doc claim of "equivalent to X" is load-bearing and warrants a backing test. builder, fixer, weaver cite rename-discipline; builder and fixer's existing changeset-discipline and regression-evidence citations gain one-line annotations for the new content. CLAUDE.md inventory adds rename-discipline.
Address upstream review feedback on PR #3232 by exposing the
keystream's internal state for introspection and resumption.
`makeChaCha12(key)` now returns a `ChaCha12Generator` record
`{ next, getState, clone, fillRandomBytes }` instead of a bare
`(out: Uint8Array) => void` function. The new shape is structurally
compatible with `pure-rand` v8's `RandomGenerator` (the contract
`fast-check@4` consumes via its `randomType` parameter):
- `next()` returns a signed int32 in `[-0x80000000, 0x7fffffff]`
by reading 4 little-endian keystream bytes.
- `getState()` returns a 34-element `readonly number[]` snapshot
`[base0..base15, counter, offset, block0..block15]`.
- `clone()` returns a fully independent generator at the same
position (not an alias).
- `fillRandomBytes(out)` preserves the original byte-keystream
entry point and conforms to `@endo/random`'s `RandomSource`.
Add `makeChaCha12FromState(state)` for deterministic resumption
from a snapshot, completing the pure-rand-style paired
`makeXxx` / `makeXxxFromState` factory convention.
Update in-tree consumers (`@endo/random`, `@endo/hex`,
`@endo/ocapn`) to destructure `fillRandomBytes` from the returned
record. The migration is mechanical:
-const fillRandomBytes = makeChaCha12(seed);
+const { fillRandomBytes } = makeChaCha12(seed);
Add a new private package `@endo/chacha12-fast-check-test` that
drives `fast-check@4`'s property-based tester through the new
chacha12 surface as its `randomType` source. The package depends
on `fast-check@^4` directly; the production-shaped fast-check /
pure-rand adapters remain in the separately-designed
`@endo/random-fast-check` package.
The new chacha12 unit tests cover the int32 contract for `next`,
the round-trip property for `getState` /
`makeChaCha12FromState` (across every offset 0..64 to catch
block-boundary off-by-ones), `clone` independence, and malformed
state rejection. 18 chacha12 tests + 7 fast-check integration
tests pass.
Refs: endojs/endo#3232 (comment)
Refs: #75
Per upstream review comments 3239081576 and 3239082370 on endojs/endo#3232: the module-level callable was better when named `random`. The earlier rename to `randomNumber` was a side-effect of importing `random as randomFloat`; the imported `random` does not appear in the module's value namespace, so there is no shadowing and the local `random` identifier is unambiguous.
Per upstream review comments on endojs/endo#3232: - 3239072009: fold chacha12-next-getstate.md into endo-chacha12.md. We have only made one release-cycle change with this PR; a single changeset captures it without the synthetic before/after framing. - 3239062983: omit the gratuitous process commentary (the "### Why / asked for ... / new sibling package exercises the surface" section). - 3239068864: omit the "Rewires the @endo/hex / @endo/ocapn ..." paragraph; that is not interesting to package authors updating dependencies (the @endo/hex and @endo/ocapn shipped runtime surfaces are unchanged; only their dev fixtures were rewired). - 3239067688: rewrite the @endo/chacha12 interface description to reflect the actual landed shape (ChaCha12Generator record with fillRandomBytes satisfying RandomSource, plus next/getState/clone for pure-rand v8 structural compatibility), not the stale "bare (out) => void" shape. - 3239064618: drop the "pure-rand v5" wording. We are structurally compatible with pure-rand v8 and fast-check v4 now.
|
Force-pushed to incorporate three new fixups from the source branch since the prior tip:
The prior tip |
Address upstream review feedback on PR #3232 by exposing the
keystream's internal state for introspection and resumption.
`makeChaCha12(key)` now returns a `ChaCha12Generator` record
`{ next, getState, clone, fillRandomBytes }` instead of a bare
`(out: Uint8Array) => void` function. The new shape is structurally
compatible with `pure-rand` v8's `RandomGenerator` (the contract
`fast-check@4` consumes via its `randomType` parameter):
- `next()` returns a signed int32 in `[-0x80000000, 0x7fffffff]`
by reading 4 little-endian keystream bytes.
- `getState()` returns a 34-element `readonly number[]` snapshot
`[base0..base15, counter, offset, block0..block15]`.
- `clone()` returns a fully independent generator at the same
position (not an alias).
- `fillRandomBytes(out)` preserves the original byte-keystream
entry point and conforms to `@endo/random`'s `RandomSource`.
Add `makeChaCha12FromState(state)` for deterministic resumption
from a snapshot, completing the pure-rand-style paired
`makeXxx` / `makeXxxFromState` factory convention.
Update in-tree consumers (`@endo/random`, `@endo/hex`,
`@endo/ocapn`) to destructure `fillRandomBytes` from the returned
record. The migration is mechanical:
-const fillRandomBytes = makeChaCha12(seed);
+const { fillRandomBytes } = makeChaCha12(seed);
Add a new private package `@endo/chacha12-fast-check-test` that
drives `fast-check@4`'s property-based tester through the new
chacha12 surface as its `randomType` source. The package depends
on `fast-check@^4` directly; the production-shaped fast-check /
pure-rand adapters remain in the separately-designed
`@endo/random-fast-check` package.
The new chacha12 unit tests cover the int32 contract for `next`,
the round-trip property for `getState` /
`makeChaCha12FromState` (across every offset 0..64 to catch
block-boundary off-by-ones), `clone` independence, and malformed
state rejection. 18 chacha12 tests + 7 fast-check integration
tests pass.
Refs: endojs/endo#3232 (comment)
Refs: #75
Per upstream review comments 3239081576 and 3239082370 on endojs/endo#3232: the module-level callable was better when named `random`. The earlier rename to `randomNumber` was a side-effect of importing `random as randomFloat`; the imported `random` does not appear in the module's value namespace, so there is no shadowing and the local `random` identifier is unambiguous.
Per upstream review comments on endojs/endo#3232: - 3239072009: fold chacha12-next-getstate.md into endo-chacha12.md. We have only made one release-cycle change with this PR; a single changeset captures it without the synthetic before/after framing. - 3239062983: omit the gratuitous process commentary (the "### Why / asked for ... / new sibling package exercises the surface" section). - 3239068864: omit the "Rewires the @endo/hex / @endo/ocapn ..." paragraph; that is not interesting to package authors updating dependencies (the @endo/hex and @endo/ocapn shipped runtime surfaces are unchanged; only their dev fixtures were rewired). - 3239067688: rewrite the @endo/chacha12 interface description to reflect the actual landed shape (ChaCha12Generator record with fillRandomBytes satisfying RandomSource, plus next/getState/clone for pure-rand v8 structural compatibility), not the stale "bare (out) => void" shape. - 3239064618: drop the "pure-rand v5" wording. We are structurally compatible with pure-rand v8 and fast-check v4 now.
Per upstream review comment 3239085874 on endojs/endo#3232: add an assertion to the test suite that captures the float extraction recipe `random(source) === randomUint53(source) * 2 ** -53` so a future implementation edit cannot silently drift the multiplier off the magic constant. The test constructs three fixed-byte mock RandomSources whose randomUint53 returns 1, 2**53 - 1, and a chosen mid-range value, and asserts each random() output equals the integer times 2 ** -53 exactly.
| // Pin the magic multiplier to exactly `2 ** -53`. When randomUint53 | ||
| // returns 1, random() returns the multiplier itself, so this asserts | ||
| // the constant in `src/random.js` is what its accompanying comment | ||
| // claims it is. | ||
| test('random() multiplies randomUint53 by exactly 2 ** -53', t => { | ||
| /** @param {Uint8Array} out */ | ||
| const oneSource = out => { | ||
| for (let i = 0; i < out.length; i += 1) out[i] = 0; | ||
| out[0] = 1; | ||
| }; | ||
| t.is(random(oneSource), 2 ** -53); | ||
| }); |
There was a problem hiding this comment.
This strikes me as overly brittle; it presumes a particular pattern of consumption that is not required (i.e., little-endian consumption of the buffer that retains its first octet and interprets it as the least significant of the value to be scaled down). Less brittle would be something like this:
| // Pin the magic multiplier to exactly `2 ** -53`. When randomUint53 | |
| // returns 1, random() returns the multiplier itself, so this asserts | |
| // the constant in `src/random.js` is what its accompanying comment | |
| // claims it is. | |
| test('random() multiplies randomUint53 by exactly 2 ** -53', t => { | |
| /** @param {Uint8Array} out */ | |
| const oneSource = out => { | |
| for (let i = 0; i < out.length; i += 1) out[i] = 0; | |
| out[0] = 1; | |
| }; | |
| t.is(random(oneSource), 2 ** -53); | |
| }); | |
| // We expect random() to achieve a uniform distribution by scaling down | |
| // randomUint53() output by exactly `2 ** -53`, but don't want to overly | |
| // constrain _how_ the former maps octets into integers (consuming 8 vs. 7, big | |
| // vs. little endian, etc.). So we provide a source that sets every bit to | |
| // force maximum output, and if we ever need even more assurance could also use | |
| // sources that set every bit of four contiguous octets after a prefix of 0 to 4 | |
| // fully-clear octets (0xFFFFFFFF_00..., 0x00FFFFFF_FF00..., | |
| // 0x0000FFFF_FFFF00..., 0x000000FF_FFFFFF00..., 0x00000000_FFFFFFFF_00...), | |
| // exactly one of which should drive random() output to be `2 ** -21 - 2 ** -53` | |
| // (i.e., the one in which the set octets correspond exactly with the least | |
| // significant 32 bits for randomUint53). | |
| test('random() multiplies randomUint53 by exactly 2 ** -53', t => { | |
| /** @param {Uint8Array} out */ | |
| const maxSource = out => { | |
| for (let i = 0; i < out.length; i += 1) out[i] = 0xFF; | |
| }; | |
| t.is(random(maxSource), 1 - 2 ** -53); | |
| }); |
There was a problem hiding this comment.
But on the other hand, this brittleness could be construed as a feature that prevents refactors from accidentally slipping in breaking changes. In which case I would highlight that in the explanatory comments (but also still do the set-all-bits analog as well).
There was a problem hiding this comment.
Agreed. This was emitted in order to provide assurance that the max float int constant matched the value prescribed in the comment, but I like narrowing this fence. I’m going to propose both all set, none set, 52 bits set, and 53 bits set.
…3 (per r3245918820) Replace the brittle `oneSource` test (which presumed little-endian octet consumption with a retained-first-octet pattern) with the `maxSource` form gibson042 suggested in endojs/endo#3232 (review comment r3245918820): set every byte to `0xFF` so randomUint53 returns `2 ** 53 - 1`, then assert `random(maxSource) === 1 - 2 ** -53`. This pins the multiplier without coupling the test to any specific octet-to-integer mapping inside randomUint53 (8 vs 7 octets, big vs little endian).
…3 (per r3245918820) Replace the brittle `oneSource` test (which presumed little-endian octet consumption with a retained-first-octet pattern) with the `maxSource` form gibson042 suggested in endojs/endo#3232 (review comment r3245918820): set every byte to `0xff` so randomUint53 returns `2 ** 53 - 1`, then assert `random(maxSource) === 1 - 2 ** -53`. This pins the multiplier without coupling the test to any specific octet-to-integer mapping inside randomUint53 (8 vs 7 octets, big vs little endian). The hex literal is lowercased (`0xff`) to satisfy prettier's default; gibson042's verbatim suggestion used `0xFF`, but lowercase is the package-wide convention and prettier's formatting rule.
…3 (per r3245918820) Replace the brittle `oneSource` test (which presumed little-endian octet consumption with a retained-first-octet pattern) with the `maxSource` form gibson042 suggested in endojs/endo#3232 (review comment r3245918820): set every byte to `0xff` so randomUint53 returns `2 ** 53 - 1`, then assert `random(maxSource) === 1 - 2 ** -53`. This pins the multiplier without coupling the test to any specific octet-to-integer mapping inside randomUint53 (8 vs 7 octets, big vs little endian). The hex literal is lowercased (`0xff`) to satisfy prettier's default; gibson042's verbatim suggestion used `0xFF`, but lowercase is the package-wide convention and prettier's formatting rule.
kriskowal
force-pushed
the
kriskowal-random-chacha20
branch
from
6fbe4b0 to
71055ef
Compare
…(base-drift refresh; clear CONFLICTING; random/chacha12)
A source-agnostic library of random sampling functions (`random`, `randomInt`, plus the underlying `randomUint8`, `randomUint16`, `randomUint24`, `randomUint32`, `randomUint53` readers). Each function accepts a `RandomSource`: `(out: Uint8Array) => void`, matching `crypto.getRandomValues` minus the return value. Names follow the TC39 proposal-random-functions translation `Random.method` -> `randomMethod`. Each sampler is its own module so consumers can import only what they use. Also ships `@endo/random/seeds.js`, exporting the canonical `bobsCoffee32` 32-byte seed shared across Endo deterministic fuzz suites.
A pure-JavaScript ChaCha12 keystream. The factory
`makeChaCha12(key)` returns a `ChaCha12Generator` record
`{ next, getState, clone, fillRandomBytes }`. The
`fillRandomBytes` method has the shape
`(out: Uint8Array) => void`, conforming to
`@endo/random`'s `RandomSource` and to
`crypto.getRandomValues`-style ergonomics; it can be passed
directly to the samplers.
The remaining methods expose the keystream's internal state
(snapshot via `getState`, independent copy via `clone`,
signed-int32 pull via `next`) so the generator satisfies
`pure-rand` v8's `RandomGenerator` interface structurally,
driving `fast-check` v4's `randomType` parameter without a
separate adapter. A companion `makeChaCha12FromState(state)`
reconstructs a generator from a snapshot for deterministic
resumption.
The keystream is cross-checked against three published
ChaCha12 test vectors from
`draft-strombergson-chacha-test-vectors-01` (TC1, TC4, TC8).
A companion benchmark (`fill-random-bytes.bench.js`) measures
keystream throughput across engines.
A companion test package that exercises `@endo/chacha12`'s structural `pure-rand` v8 `RandomGenerator` shape by driving `fast-check` v4's `randomType` parameter. Adopting the test-package shape (separate package with its own dev-deps, no `main`, tests under `test/`) keeps `fast-check` out of `@endo/chacha12`'s declared surface while still proving the interface compatibility under CI.
…bench inputs Drop the in-tree `_xorshift.js` copy in favor of `@endo/chacha12`'s `fillRandomBytes` seeded from `@endo/random/seeds.js`'s `bobsCoffee32`. The shared seed keeps the bench's deterministic input identical across the hex and ocapn fuzz suites without duplicating the PRNG implementation in each test directory.
Drop the in-tree `_xorshift.js` copy in favor of `@endo/chacha12`'s `fillRandomBytes` driving `@endo/random/random.js`'s `random` (the `[0, 1)` float sampler). The shared default seed (`bobsCoffee32`) keeps the fuzz determinism identical across the hex and ocapn suites without duplicating the PRNG implementation per test directory.
…ons> overlap The original `...CompartmentOptionsArgs` rest signature produced a single-element tuple of unions, which `Parameters<typeof compartmentOptions>` (declared as `CompartmentOptionsArgs`, no rest) did not overlap. The typedoc/typecheck pass surfaced the mismatch. Re-typing `args` as the union itself (with the `@this` annotation collapsed into the same JSDoc block) restores the overlap.
Capture why a top-level `.js` thunk module exists (legacy `exports`-map portability and public-interface filtering) and when it can be removed. Surfaced by the `@endo/random` package introduction, whose multi-entry shape forced the question of whether each public sampler needed a thunk versus pointing `exports` at `src/` directly.
…config and typedoc Wire the three new packages into the workspace's composite tsconfig so `yarn build` resolves them, and exclude `chacha12-fast-check-test` from typedoc generation (test-package shape: no public docs surface).
Announce `@endo/random` (major) and `@endo/chacha12` (major) as new packages, and the `@endo/hex` and `@endo/ocapn` (patch) test-driver swaps from the in-tree xorshift copies.
kriskowal
force-pushed
the
kriskowal-random-chacha20
branch
from
71055ef to
46e330a
Compare
…e-drift refresh; CONFLICTING->MERGEABLE)
…ndojs/endo#3232 (numeric-sep autofix + SECURITY.md)
Run `yarn lint --fix` to insert numeric separators in the documented 3-digit-group style on all numeric literals across the random and chacha12 packages. Tests for both packages continue to pass; the rewrites preserve value semantics.
The package-uniformity check requires per-package SECURITY.md to be byte-identical to the skel template. The newly added chacha12 and random packages shipped a copy that already spelled "GitHub" with the canonical capitalization; the skel template still reads "Github". Sync the packages to the skel until the skel is itself updated. Lint now passes end to end.
…46e330a2b..cc336d40a)
…/endo#3232; flag cross-link pagination false-negative
3 participants
Description
Adds two new workspace packages and rewires the existing
deterministic-fuzz consumers in this repo to use them.
@endo/randomA source-agnostic library of random sampling functions. Each
sampler accepts a
RandomSourceas its first argument:The shape mirrors
crypto.getRandomValues(minus the return value),so the canonical browser/Node entropy source and a
@endo/chacha12-backed source returned bymakeChaCha12(seed)areboth directly usable wherever a
RandomSourceis expected.Names follow the TC39
proposal-random-functions
translation
Random.method->randomMethod. Each sampler is itsown subpath export so consumers can import only what they use:
@endo/random@endo/random/random.jsrandom@endo/random/int.jsrandomInt@endo/random/uint.jsrandomUint8,randomUint16,randomUint24,randomUint32,randomUint53@endo/random/fast-check.jsadaptToPureRandomGenerator,adaptFromPureRandomGenerator,makeRandomTypeFromSeed@endo/random/seeds.jsbobsCoffee64@endo/random/fast-check.jscarries bidirectional adapters betweenRandomSourceand thepure-randv5RandomGeneratorinterfacethat
fast-checkconsumes via therandomTypeparameter, plus amakeRandomTypeFromSeedbuilder. It imports nothing frompure-randorfast-check; it depends only on those interfacesbeing structurally compatible.
@endo/random/seeds.jsexportsbobsCoffee64, a 32-byte seed(
b0b5c0ffeefacaderepeated four times) shared across Endodeterministic fuzz suites so a single seed change propagates.
@endo/chacha12A pure-JavaScript ChaCha12 keystream. The package's sole public
entry point is
makeChaCha12(key), which returns a function(out: Uint8Array) => voidthat fillsoutwith the next bytes ofthe keystream. That shape matches
crypto.getRandomValuesand@endo/random'sRandomSourceexactly, so the same function serveseither ecosystem and the two packages compose without an adapter.
ChaCha12 is the 12-round variant of Daniel J. Bernstein's ChaCha
family. The block function is identical to ChaCha20 modulo the
loop count (6 double-rounds vs 10). The reduced round count trades
cryptographic safety margin for speed. The package is positioned
and documented as a PRNG, not a cipher recommendation.
Consumer rewires
The
@endo/hexencode/decode benchmarks and the@endo/ocapnsyrup and passable fuzz tests now draw deterministic bytes from
@endo/chacha12directly, seeded from@endo/random/seeds.js'sbobsCoffee64. This removes the duplicated_xorshift.jshelpersthat previously lived in
packages/hex/test/andpackages/ocapn/test/.Most critical files to review:
packages/random/src/int.js— the range-aware rejection-samplingstaircase for
randomInt. Each tier picks the smallest drawwidth that covers the requested range with reject fraction
below 0.5, capping the worst-case reject rate across the entire
safe-integer domain.
packages/random/src/read-uint.js— fixed-width little-endianunsigned readers backed by per-width scratch buffers (sharing a
single 8-byte buffer measured ~2.5x slower under V8's
element-kind specialization). Each reader zeroes its scratch
prefix after the read.
packages/random/src/random.js—random()returnsrandomUint53(source) * 2 ** -53, deterministic across runs andengines.
packages/random/fast-check.js—pure-randv5RandomGeneratoradapters and the
randomTypebuilder. Documents theclone()alias semantics for serial keystreams without a snapshot facility.
packages/random/seeds.js— canonicalbobsCoffee64seed.packages/random/random.types.d.ts—RandomSourceandPureRandomGeneratortype definitions.packages/chacha12/src/chacha12.js— the block function, statelayout, and the IETF / RFC 7539 nonce convention.
packages/chacha12/test/chacha12.test.js— TC1, TC4, TC8keystream vectors from
draft-strombergson-chacha-test-vectors-01, with a layout notedocumenting the DJB 8-byte-IV vs IETF 12-byte-nonce
reconciliation at counter = 0.
packages/random/test/random.bench.js— the cross-sourcemicrobenchmark; lives in
@endo/randomrather than@endo/chacha12because it drives both keystreams through thesame samplers (and because chacha12 cannot devDep on random
without a cycle — random already devDeps on chacha12).
packages/hex/test/{decode,encode}.bench.js,packages/ocapn/test/{codecs/passable-fuzz,syrup/fuzz}.test.js—consumer-side rewires.
Security Considerations
@endo/chacha12is positioned and documented as a PRNG, not acipher. The README explicitly steers cipher use cases to ChaCha20
or another 20-round implementation; the
cipherpackage keywordhas been intentionally omitted. ChaCha12 reduces the cryptographic
safety margin relative to ChaCha20 (12 rounds vs 20); for
deterministic test fixtures, property-based testing, and fuzzing —
the consumer use cases in this PR — the extra speed is the right
tradeoff, but downstream users should not adopt this package for
confidentiality or for key derivation from caller-supplied seeds.
Neither package introduces new authority surface.
makeChaCha12is a pure deterministic function of its 32-byte key; the returned
function is hardened.
@endo/random's samplers are pure functionsover their
sourceargument, with module state limited to smallper-width scratch buffers that are zeroed after each read so no
random bytes linger across calls. No I/O, no entropy source, no
global state.
The hex/ocapn rewires add a workspace
devDependencyonly — noruntime dependency change for consumers of those packages.
Scaling Considerations
@endo/chacha12/BENCH.mdreports the cross-source microbenchmark(
packages/random/test/random.bench.js). Headline numbers on thetest bed (Node 22.22.2, AMD Ryzen AI MAX+, x64, median of 10 runs):
random()(ns/call)randomInt(0, 99)(ns/call)The bulk-bytes ratio is below the naive 20/12 = 1.67x because
per-block fixed costs are identical between ChaCha20 and ChaCha12.
The sampler workloads narrow the gap further because the per-call
envelope (range-aware staircase, mask-and-reject) is shared across
all sources and amortizes the keystream difference.
No new on-chain storage, message exchange, or persistent resource
consumption.
Documentation Considerations
Self-contained docs ship with each new package:
packages/random/README.md—RandomSourceinterface, subpathexports, determinism guarantees, scratch-buffer rationale.
packages/chacha12/README.md—makeChaCha12API, ChaCha12 vsChaCha20 framing, keystream-length bound, verification.
packages/chacha12/BENCH.md— full benchmark report.packages/random/SECURITY.md,packages/chacha12/SECURITY.md—standard Endo security policy.
.changeset/endo-chacha12.md—@endo/random: minor,@endo/chacha12: minor,@endo/hex: patch,@endo/ocapn: patch.No existing data or deployment is affected — both packages are new,
and the hex/ocapn changes are confined to test/benchmark helpers.
No upgrade instructions are needed for downstream users of
@endo/hexor@endo/ocapn.Testing Considerations
packages/random/test/:random.test.js—random(source)range invariants,determinism, and a pinned 4-element golden float vector.
int.test.js— closed-interval invariants, endpoint coverage,bounds and unsafe-range error paths, determinism, and a
draw-width sweep that exercises every tier of the
rejection-sampling staircase (1-byte through 53-bit slow path).
pure-rand.test.js—adaptToPureRandomGenerator/adaptFromPureRandomGeneratorround-trip equivalence with afresh twin, plus distribution-shape sanity.
fast-check.test.js— drivesfc.assertwith aChaCha12-backed
randomTypeto confirm the adapter satisfiesfast-check's contract end-to-end.random.bench.js— local microbenchmarks against an inlinedChaCha20 reference (
_chacha20.js) and anxorshift128+baseline (
_xorshift.js).packages/chacha12/test/chacha12.test.js— three publishedChaCha12 keystream vectors (TC1, TC4, TC8) from
draft-strombergson-chacha-test-vectors-01, plus block-boundaryequivalence (a 192-byte single pull vs piecewise 7/57/64/32/32
pulls from a freshly-seeded twin) and
RandomSource-shapeconformance.
The
@endo/hexbenchmarks and@endo/ocapnfuzz tests now runagainst the new chacha12 source; existing assertions are preserved.
No new CI jobs required.
Compatibility Considerations
@endo/randomand@endo/chacha12are new packages — no priorusage to break.
@endo/hexand@endo/ocapnchange only test and benchmarkinternals; their published runtime surface is unchanged. The
duplicated
_xorshift.jshelpers were never exported.Upgrade Considerations
Pure additive change for production systems:
@endo/hexor@endo/ocapnconsumers (the
@endo/chacha12and@endo/randomlinks aredevDependencies).