diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index d0afa30fb..9b8db716e 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -10,7 +10,7 @@ Plan: `.claude/plans/d-relalg-unified-v1.md`. Decisions recorded there: ADMISSIO | **D-RU-3** | cross-rule CSE in crepe-plan-probe (256-rule benchmark, separate from the hop benchmark) | Queued (after D-RU-2) | answers == crepe and ascent; traversals and materializations measured | | **D-RU-4** | co-addressed join operator over `JoinDomain` | Queued (after D-RU-2) | parity with the star path; no position taken from another table's values | | **D-RU-5** | Quack physical pins (dense vs sorted, materialize vs recompute) | Queued (after D-RU-3/4) | can-fire and can-stay-silent per pin | -| **D-RU-6** | recursion driver over BIND tiles (256-hop benchmark) | Queued/Lab (slot assignment must be argued as an attested frozen-codebook lookup under ADMISSION = B). Lab probe MEASURED (`crepe-plan-probe/examples/walk_tile.rs`, `entries/2026-10-21-d-ru-6-tile-walk-probe.md`): tuple sets equal ascent, crepe and walk_mask; execution gather-free; binding witness probe-local, so ADMISSION B is NOT demonstrated; count-only 1.0–9.3× slower than walk_mask, end to end 1.1–2.0× slower. Block-sparse arm C MEASURED (`examples/walk_sparse.rs`, `entries/2026-10-21-d-ru-6-sparse-compute.md`): faster than the dense tile path on every fixture (0.11–0.96× count-only); against walk_mask end to end it wins only on deep low-fan-out recursion (chain(230) 0.82×, chain(700) 0.59× UNCHECKED), is near parity on sparse graphs (1.03–1.07×) and loses on dense, cyclic, clustered and mixed-length graphs (1.20–2.43×) | least fixpoint == datafrog; ≥ 200 iterations asserted; cycles terminate. Benchmark requirement (operator, 2026-10-21; refs D-RMAP-2 §5b, #1642): tuple materialization, count-only evaluation, memory growth, address binding and value-derived reads by origin reported separately; tuple-set equality with crepe and ascent wherever they run | +| **D-RU-6** | recursion driver over BIND tiles (256-hop benchmark) | Queued/Lab (slot assignment must be argued as an attested frozen-codebook lookup under ADMISSION = B). Lab probe MEASURED (`crepe-plan-probe/examples/walk_tile.rs`, `entries/2026-10-21-d-ru-6-tile-walk-probe.md`): tuple sets equal ascent, crepe and walk_mask; execution gather-free; binding witness probe-local, so ADMISSION B is NOT demonstrated; count-only 1.0–9.3× slower than walk_mask, end to end 1.1–2.0× slower. Block-sparse arm C MEASURED (`examples/walk_sparse.rs`, `entries/2026-10-21-d-ru-6-sparse-compute.md`): faster than the dense tile path on every fixture (0.11–0.96× count-only); against walk_mask end to end it wins only on deep low-fan-out recursion (chain(230) 0.82×, chain(700) 0.59× UNCHECKED), is near parity on sparse graphs (1.03–1.07×) and loses on dense, cyclic, clustered and mixed-length graphs (1.20–2.43×). Arm A′ (walk_mask with a dirty-frontier skip, `examples/walk_dirty.rs`, `entries/2026-10-21-d-ru-6-dirty-frontier.md`) does not remove C's chain lead; the chain sweep (two runs, distinct fan-out, like-for-like timing) confines C's end-to-end win over A′ to unbranched chains of about 192 nodes or more (3–30 %); with fan-out 2 or 4 C loses end to end at most points. Recommendation: defer the codebook attestation | least fixpoint == datafrog; ≥ 200 iterations asserted; cycles terminate. Benchmark requirement (operator, 2026-10-21; refs D-RMAP-2 §5b, #1642): tuple materialization, count-only evaluation, memory growth, address binding and value-derived reads by origin reported separately; tuple-set equality with crepe and ascent wherever they run | | **D-RU-7** | prefetch and staging of Arm B ranges | Queued (after D-RU-6 or a non-recursive multi-stage plan) | critical-path overlap measured; bytes read unchanged | | **D-RU-8** | Wikidata- and SAP-shaped fixtures with completeness refusals | Queued (after D-RU-6/7) | `!R` refused under an incomplete scope | diff --git a/.claude/board/entries/2026-10-21-d-ru-6-dirty-frontier.md b/.claude/board/entries/2026-10-21-d-ru-6-dirty-frontier.md new file mode 100644 index 000000000..44402c280 --- /dev/null +++ b/.claude/board/entries/2026-10-21-d-ru-6-dirty-frontier.md @@ -0,0 +1,109 @@ +# 2026-10-21 — D-RU-6-LAB-3: walk_mask with a dirty-frontier skip (arm A′), and the chain sweep + +**Status:** MEASURED, lab only. D-RU-6 stays **Queued/Lab**. A′ is a gather path, like A, so it is not an ADMISSION-B candidate. It tests what causes arm C's chain win. It is not a production option. Follows D-RU-6-LAB-2 (`entries/2026-10-21-d-ru-6-sparse-compute.md`, #1648). Order set by the operator (EXO-50, 2026-10-21): +1. A′ first. +2. If C still wins against A′ on oracle-checked chains, a sweep over chain length, fan-out and number of foci. +3. Codebook attestation only if a convincing region remains after the sweep. + +## What landed + +`crates/crepe-plan-probe/examples/walk_dirty.rs`, a fourth arm. It compiles `walk_sparse.rs`, which compiles B and A, so all four arms run on the same graphs with the same oracles. A is unchanged. `walk_mask.rs`, `walk_tile.rs` and `walk_sparse.rs` gain only `pub(crate)`. + +A′ is A's loop with two switches. Each switch can be measured on its own: +- **`skip_clean`.** Each delta plane carries a bitmap of its non-zero rows. Every adjacency word is masked with it before the scan, so a set bit whose delta row did not change costs nothing. A only skips whole lengths with nothing new. +- **`touched_only`.** A clears the whole `new` buffer every iteration. It also runs `andnot`, `any` and `or` over the full `n × w` span of each live length. With this switch, only the rows the hop wrote are updated, and only the rows that held a delta are cleared. + +With both switches off, A′ is A. A test asserts equal tuples and an equal count of value-addressed row reads. + +**Tests (9, plus 38 of A, B and C):** +- tuple-set equality with ascent, crepe and A for all four switch settings; +- mixed lengths with cycles; alternative path lengths; a zero-length ring with over-length edges; a 300-node chain across word boundaries; clustered, dense and cyclic fixtures; the sweep generator over two foci, where C is also checked; the sweep generator builds distinct successors (exactly `fan` per node away from the tail, no duplicate edge); +- `skip_clean` masks bits on a chain, its row reads plus masked bits equal A's, and `touched_only` sweeps under a tenth of A's words; +- the silence twin: an all-dirty frontier masks nothing. + +**Disable runs** (committed first, each reverted, all red): + +| Disable | Red | +|---|---| +| dirty bitmap never refreshed | 7 tests | +| written rows not recorded | 7 tests | +| skip mask read from the wrong length | 7 tests | +| the "off" variant secretly skips | 6 tests | +| length bound made exclusive | 3 tests | +| old delta rows not cleared | the cleared-buffer check | +| extra successors drawn with replacement | the distinct-successor test | + +A missing row clear cannot change the answer: a stale bit is already in `walk`, and `new &= !walk` removes it. So the first run of that disable stayed green. A test-only assertion that `new` is clear at the start of every iteration now catches it. + +CI runs `cargo test --example walk_dirty`. The benchmark and the sweep are not run in CI. + +## Measured (release, 4 vCPU, load ≈ 1, medians of 5) + +**Correction (codex review on #1652).** The first version of this entry had three faults, fixed before these numbers: +- **End-to-end timing was not like for like.** For A and A′ it included the count pass; for B and C (`e2e()`) it did not. A and A′ end to end now time the fixpoint without counts, plus decoding. +- **The sweep's fan-out was overstated.** `chain_fan` drew extra successors with replacement, so fan 2 and 4 contained duplicate edges. It now draws without replacement, and the sweep reports the distinct out-degree it built: 1.00, 1.99–2.00 and 3.95–3.99. +- **The claimed cutoff contradicted the entry's own data.** "C wins only on unbranched chains of ≥ 384 nodes" conflicted with the checked chain(230) result. The sweep now samples 128, 192, 230, 256, 320, 384, 512 and 768 nodes, and it was run twice. + +**Count-only, ms:** + +| fixture | A | A′ skip | A′ touched | A′ both | C | A′both/A | C/A′both | +|---|---:|---:|---:|---:|---:|---:|---:| +| sparse(512) | 26.3 | 33.2 | 28.4 | 28.3 | 24.4 | 1.08× | 0.86× | +| clustered(512) | 25.3 | 28.7 | 24.7 | 34.9 | 58.2 | 1.38× | 1.67× | +| dense(128) | 2.95 | 2.29 | 3.57 | 2.77 | 6.15 | 0.94× | 2.22× | +| cyclic(300) | 213 | 279 | 434 | 413 | 880 | 1.94× | 2.13× | +| chain(230) | 5.88 | 5.74 | 7.60 | 4.83 | 4.37 | 0.82× | 0.90× | +| chain(700), UNCHECKED | 135 | 123 | 79.0 | 64.6 | 35.3 | 0.48× | 0.55× | +| mixed(256) | 9.97 | 10.2 | 16.1 | 21.1 | 116 | 2.12× | 5.48× | +| edges(512) | 3.57 | 3.86 | 2.37 | 2.27 | 5.27 | 0.64× | 2.32× | +| edges(1024), UNCHECKED | 15.8 | 16.2 | 7.44 | 8.65 | 35.2 | 0.55× | 4.06× | + +**End to end, tuples decoded, C/A′both** (C/A in brackets): +- chain(230) 0.79× (0.77×); chain(700) 0.82× (0.60×), UNCHECKED; +- sparse(512) 0.91× (1.12×); edges(512) 1.10× (1.00×); edges(1024) 1.07× (0.84×), UNCHECKED; +- dense 1.25×, mixed 1.66×, clustered 1.78×, cyclic 2.10×. + +**Work at chain(700):** +- C's row ORs: 225,650. A′ with `skip_clean` makes exactly 225,650 value-addressed row reads, against A's 669,729. +- A′ still scans 14.6 M adjacency words and sweeps 9.9 M words; A sweeps 230 M. +- C and A′ therefore do the same row operations. What remains is A′ scanning every adjacency row of every live length, and A′'s rows being 11 words wide against a tile row's 4. + +**Memory:** A′'s count-only resident growth is at or below A's where the planes are mostly empty (3.0 against 14.3 MiB at edges(1024)), and slightly above A's on small cyclic or dense graphs. + +### Chain sweep, C/A′ end to end (two runs: run A / run B) + +`--sweep`: a chain of `len` nodes; each node gets `fan - 1` extra edges to distinct nodes 2–8 ahead. Up to 512 nodes every point is checked against ascent and crepe; 768 is UNCHECKED. Foci: 1 up to 256 nodes, 2 for 320–512, 3 for 768. + +| len | fan 1 | fan 2 | fan 4 | +|---:|---:|---:|---:| +| 128 | 1.25 / 1.08 | 1.19 / 1.04 | 1.34 / 1.36 | +| 192 | 0.95 / 0.97 | 1.10 / 1.21 | 0.94 / 1.05 | +| 230 | 0.95 / 0.75 | 0.91 / 0.97 | 1.12 / 0.87 | +| 256 | 1.01 / 0.94 | 1.48 / 0.91 | 1.07 / 1.07 | +| 320 | 0.81 / 0.86 | 1.09 / 1.02 | 1.19 / 1.18 | +| 384 | 0.72 / 0.94 | 1.02 / 1.07 | 1.11 / 1.10 | +| 512 | 0.78 / 0.90 | 1.31 / 1.27 | 1.24 / 1.19 | +| 768 (UNCHECKED) | 0.75 / 0.70 | 1.04 / 1.01 | 1.09 / 1.14 | + +Count-only, C leads A′ more often: fan 1 from 192 nodes on (0.46–1.04×), and fan 4 from 192 nodes on except at 512 (0.60–1.00×; 1.14–1.16× at 512). Single cells move by up to 0.3 between runs (256/2: 1.48 against 0.91), so only patterns that hold in both runs are read below. + +## Reading + +- **A′ against A.** Both switches together make walk_mask 1.2× faster on chain(230), 2.1× on chain(700) and 1.6–1.8× on the benchmark generator. They make it 1.4–2.1× slower on clustered, cyclic and mixed-length graphs, and 1.08× slower on sparse(512), where the bookkeeping does not pay. On chain(230) neither switch alone helps by more than 2 %. On chain(700) `touched_only` alone helps (79.0 against 135 ms) and both together help most. +- **A′ does not remove C's chain lead.** On chain(230) and chain(700), C stays ahead of A′ count-only and end to end, while doing the same row operations. +- **Where C wins end to end against A′ (both runs agree):** unbranched chains (fan 1) from 192 nodes upward, by 3–30 %. At 256 nodes it is at parity (1.01 / 0.94). At 128 nodes C loses. +- **With branching it mostly does not hold.** At fan 2 and 4, C loses end to end in both runs at most points (1.01–1.36×). One branched point wins in both runs: 230 nodes at fan 2 (0.91 / 0.97). Three more win in one run only: 192 and 230 at fan 4, 256 at fan 2. Decoding dominates once the graph branches, even where C wins count-only. +- **The cause is loop order, not tiles.** C's loop is driven by the changed delta blocks; A′'s visits every row `x`. A pull form of A′ (dirty `y` through a reverse adjacency) would test that directly. It is not built here. + +## Against the operator's gates + +- C stays faster than A′ on oracle-checked long chains, so the sweep ran. +- After the sweep, the end-to-end region where C wins is narrow and structural: unbranched chains of about 192 nodes or more, with a 3–30 % gain (one branched point, 230 nodes at fan 2, also wins in both runs). The earlier "≥ 384 nodes" was wrong (see the correction above); the lower edge lies between 128 and 192 nodes. +- Recommendation for the operator: **defer the codebook attestation.** No branched region survives, and the remaining gain comes from a loop-order difference that a pull form of A′ may also capture. +- D-RU-6 stays Queued/Lab. No production admission. + +## Open + +- A pull-form A′ (dirty `y` through a reverse adjacency), to test whether loop order alone accounts for C's remaining chain gain. +- Decoding cost: it dominates every branched case and none of the arms addresses it. +- Noise: two sweep runs on a quiet host, with single cells moving by up to 0.3; finer boundary claims need more repetitions. diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index 3a29ff226..6490abd61 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,13 +25,14 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -403 entries, 2026-08-06 .. 2026-10-21. +404 entries, 2026-08-06 .. 2026-10-21. | date | entry id | finding | file | |---|---|---|---| | 2026-10-21 | `D-RU-2` | | [2026-10-21-u2-let-planes.md](2026-10-21-u2-let-planes.md) | | 2026-10-21 | `D-RU-6-LAB-1` | | [2026-10-21-d-ru-6-tile-walk-probe.md](2026-10-21-d-ru-6-tile-walk-probe.md) | | 2026-10-21 | `D-RU-6-LAB-2` | | [2026-10-21-d-ru-6-sparse-compute.md](2026-10-21-d-ru-6-sparse-compute.md) | +| 2026-10-21 | `D-RU-6-LAB-3` | | [2026-10-21-d-ru-6-dirty-frontier.md](2026-10-21-d-ru-6-dirty-frontier.md) | | 2026-10-21 | `D-CMR-6` | | [2026-10-21-d-cmr-6-rule-catalogue.md](2026-10-21-d-cmr-6-rule-catalogue.md) | | 2026-10-21 | `D-CMR-5` | | [2026-10-21-d-cmr-5-rescope.md](2026-10-21-d-cmr-5-rescope.md) | | 2026-10-21 | `D-CMR-2` | | [2026-10-21-d-cmr-2-lowering.md](2026-10-21-d-cmr-2-lowering.md) | diff --git a/.claude/plans/d-relalg-unified-v1.md b/.claude/plans/d-relalg-unified-v1.md index 3b5d03da8..c839ec1a1 100644 --- a/.claude/plans/d-relalg-unified-v1.md +++ b/.claude/plans/d-relalg-unified-v1.md @@ -202,7 +202,7 @@ This is the `walk_mask` loop. To be lawful it needs the step "rule fragment appl **Benchmark requirement.** At least one input must need deep propagation. A chain or a ladder of length ≥ 200 forces the loop to run that many iterations (asserted, not assumed), plus a cyclic graph to check termination. -**Benchmark requirement, D-RU-6 (operator, 2026-10-21).** Every comparison against `walk_mask` (#1642) reports separately: tuple materialization, count-only evaluation, memory growth, address binding (attestation, witness validation, identity resolution and focus binding, projection), and value-derived reads by origin. Correctness is the full `Walk` / `NoWalk` tuple sets against crepe and ascent wherever they run, never Boolean reachability alone. D-RMAP-2 §5b's split of preparation, execution and maintenance applies. First lab measurement: `entries/2026-10-21-d-ru-6-tile-walk-probe.md` (execution gather-free; binding not shown to satisfy ADMISSION B; the tile path loses on time and wins on count-only memory). Second: `entries/2026-10-21-d-ru-6-sparse-compute.md` (a block-sparse product with loop-coordinate addressing; it beats walk_mask end to end only on deep, low-fan-out recursion). +**Benchmark requirement, D-RU-6 (operator, 2026-10-21).** Every comparison against `walk_mask` (#1642) reports separately: tuple materialization, count-only evaluation, memory growth, address binding (attestation, witness validation, identity resolution and focus binding, projection), and value-derived reads by origin. Correctness is the full `Walk` / `NoWalk` tuple sets against crepe and ascent wherever they run, never Boolean reachability alone. D-RMAP-2 §5b's split of preparation, execution and maintenance applies. First lab measurement: `entries/2026-10-21-d-ru-6-tile-walk-probe.md` (execution gather-free; binding not shown to satisfy ADMISSION B; the tile path loses on time and wins on count-only memory). Second: `entries/2026-10-21-d-ru-6-sparse-compute.md` (a block-sparse product with loop-coordinate addressing; it beats walk_mask end to end only on deep, low-fan-out recursion). Third: `entries/2026-10-21-d-ru-6-dirty-frontier.md` (walk_mask with a dirty-frontier skip; against it, C wins end to end only on unbranched chains of about 192 nodes or more). ## 9. Physical planning in Quack diff --git a/.github/workflows/crepe-plan-probe.yml b/.github/workflows/crepe-plan-probe.yml index 172f8d65e..a547172ab 100644 --- a/.github/workflows/crepe-plan-probe.yml +++ b/.github/workflows/crepe-plan-probe.yml @@ -62,6 +62,9 @@ jobs: # D-RU-6 arm C: the block-sparse product, against arms A and B and the # oracles, its refusals and the semi-naive bound. - run: cargo test --example walk_sparse + # D-RU-6 arm A′: walk_mask with the dirty-frontier switches, against A + # and the oracles. The benchmark and the chain sweep are not run in CI. + - run: cargo test --example walk_dirty # relalg is its own workspace and patches ndarray to a sibling checkout # (`../../../ndarray` from its directory); no other job runs its tests. - name: relalg tests and clippy diff --git a/crates/crepe-plan-probe/examples/walk_dirty.rs b/crates/crepe-plan-probe/examples/walk_dirty.rs new file mode 100644 index 000000000..1d066297a --- /dev/null +++ b/crates/crepe-plan-probe/examples/walk_dirty.rs @@ -0,0 +1,706 @@ +//! D-RU-6 lab probe, arm A′: `walk_mask` with two frontier optimizations, +//! each switchable on its own, measured against arms A, B and C. +//! +//! STATUS: LAB ONLY. A′ is a gather path exactly like A (it reads a delta row +//! at a position taken from an adjacency bit), so it is not an ADMISSION-B +//! candidate. It exists to find out what causes arm C's win on long chains. +//! +//! ```text +//! cargo run --release --example walk_dirty # arms A, A′ (three variants), B and C +//! cargo test --example walk_dirty # correctness and the two switches +//! cargo run --release --example walk_dirty -- --sweep # chain length x fan-out x foci +//! ``` +//! +//! # The two switches +//! +//! * **`skip_clean`.** Each delta plane carries a bitmap of its non-zero rows. +//! The hop masks each adjacency word with that bitmap before scanning it, +//! so a set bit `y` whose delta row `y` did not change this iteration costs +//! nothing. A already skips whole lengths with nothing new (`live`); this +//! skips single rows. +//! * **`touched_only`.** A clears the whole `new` buffer each iteration and +//! runs `andnot`, `any` and `or` over the full `n x w` span of every live +//! length. With this switch only the rows the hop wrote are updated, and +//! only the rows that held a delta are cleared again. +//! +//! With both switches off, A′ runs the same loop as A; a test asserts the +//! same tuple sets and the same number of value-addressed row reads. +//! +//! Counters: value-addressed row reads (the gathers), adjacency bits masked +//! out by `skip_clean`, and words swept by `fill`, `andnot`, `any`, `or` and +//! the row clears (the cost `touched_only` removes). +//! +//! Measured results: `.claude/board/entries/2026-10-21-d-ru-6-dirty-frontier.md`. + +use std::hint::black_box; +use std::time::{Duration, Instant}; + +use ndarray::simd::{mask_andnot_assign, mask_any, mask_or_assign}; + +// Arm C, which compiles arm B, which compiles arm A and the oracles. +#[allow(dead_code)] +#[path = "walk_sparse.rs"] +mod sparse; + +use sparse::tile::baseline::{ + MAX_PATH_LEN, Planes, agree, ascent_tuples, bit, crepe_tuples, mask_walk, set, words, +}; +use sparse::tile::{Edge, mib, ms, tile_eval, width}; +use sparse::{DEFAULT_M, fixtures, sparse_eval}; + +const REPS: usize = 5; +const ORACLE_MAX_NODES: usize = 512; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +struct Opt { + skip_clean: bool, + touched_only: bool, +} + +const VARIANTS: [(&str, Opt); 4] = [ + ( + "A′ off", + Opt { + skip_clean: false, + touched_only: false, + }, + ), + ( + "A′ skip", + Opt { + skip_clean: true, + touched_only: false, + }, + ), + ( + "A′ touched", + Opt { + skip_clean: false, + touched_only: true, + }, + ), + ( + "A′ both", + Opt { + skip_clean: true, + touched_only: true, + }, + ), +]; + +#[derive(Clone, Copy, Debug, Default)] +struct Counters { + iterations: usize, + /// Value-addressed row reads (`delta[row(l2, y)]` at a scanned bit `y`). + row_reads: u64, + /// Adjacency bits whose delta row was clean, skipped by `skip_clean`. + masked_out: u64, + /// Words processed by whole-buffer and per-row sweeps. + sweep_words: u64, + /// Adjacency words scanned by the hop (every row `x`, both switches). + adj_words: u64, +} + +struct Dirty { + planes: Planes, + counters: Counters, +} + +/// The program over bit planes, with the switches in `opt`. +fn dirty_walk(n: usize, e: &[Edge], opt: Opt) -> Dirty { + let w = words(n); + let lens = MAX_PATH_LEN as usize + 1; + let row = |plane: usize, x: usize| (plane * n + x) * w; + + let mut by_len: Vec>> = vec![None; lens]; + let mut node = vec![0u64; w]; + for &(a, b, l) in e { + assert!( + (0..n as i32).contains(&a) && (0..n as i32).contains(&b), + "edge ({a}, {b}) outside 0..{n}" + ); + let (a, b, l) = (a as usize, b as usize, l as usize); + set(&mut node, a); + set(&mut node, b); + if l < lens { + let adj = by_len[l].get_or_insert_with(|| vec![0u64; n * w]); + set(&mut adj[a * w..a * w + w], b); + } + } + + let mut walk = vec![0u64; lens * n * w]; + let mut delta = vec![0u64; lens * n * w]; + let mut new = vec![0u64; lens * n * w]; + let mut live = vec![false; lens]; + // Per length: the bitmap of non-zero delta rows, and the same rows as a + // list (to clear them later without a sweep). + let mut d_dirty = vec![0u64; lens * w]; + let mut d_rows: Vec> = vec![Vec::new(); lens]; + // Per length: rows of `new` the hop wrote, as a bitmap and a list. + let mut n_mark = vec![0u64; lens * w]; + let mut n_rows: Vec> = vec![Vec::new(); lens]; + for x in (0..n).filter(|&x| bit(&node, x)) { + set(&mut walk[row(0, x)..row(0, x) + w], x); + set(&mut delta[row(0, x)..row(0, x) + w], x); + set(&mut d_dirty[0..w], x); + d_rows[0].push(x); + } + live[0] = true; + + let mut c = Counters::default(); + let bound = lens * n * n + 1; + loop { + c.iterations += 1; + assert!(c.iterations <= bound, "fixpoint did not terminate"); + if !opt.touched_only { + new.fill(0); + c.sweep_words += new.len() as u64; + } + // What `touched_only` relies on instead of the fill: the rows the last + // iteration left in `new` were cleared row by row. A stale bit would + // not change the answer (it is already in `walk`, and `new &= !walk` + // removes it), so only this check can see a missing clear. + #[cfg(test)] + assert!(new.iter().all(|&x| x == 0), "`new` is not clear"); + let mut new_live = vec![false; lens]; + for (l1, adj) in by_len.iter().enumerate() { + let Some(adj) = adj else { continue }; + for l2 in (0..lens).filter(|&l2| live[l2]) { + let l = l1 + l2; + if l >= lens { + continue; + } + for x in 0..n { + let dst = row(l, x); + c.adj_words += w as u64; + for wi in 0..w { + let mut bits = adj[x * w + wi]; + if opt.skip_clean { + let clean = bits & !d_dirty[l2 * w + wi]; + c.masked_out += u64::from(clean.count_ones()); + bits &= !clean; + } + while bits != 0 { + let y = wi * 64 + bits.trailing_zeros() as usize; + bits &= bits - 1; + // The gather: a row address taken from a value. + let src = row(l2, y); + c.row_reads += 1; + mask_or_assign(&mut new[dst..dst + w], &delta[src..src + w]); + if opt.touched_only && !bit(&n_mark[l * w..l * w + w], x) { + set(&mut n_mark[l * w..l * w + w], x); + n_rows[l].push(x); + } + } + } + } + new_live[l] = true; + } + } + + // new &= !walk; walk |= new; and the next delta's dirty rows. + let mut any = false; + let mut next_dirty = vec![0u64; lens * w]; + let mut next_rows: Vec> = vec![Vec::new(); lens]; + for (l, is_live) in new_live.iter_mut().enumerate() { + if !*is_live { + continue; + } + let mut fresh = false; + if opt.touched_only { + for &x in &n_rows[l] { + let r = row(l, x); + mask_andnot_assign(&mut new[r..r + w], &walk[r..r + w]); + c.sweep_words += 2 * w as u64; + if mask_any(&new[r..r + w]) { + mask_or_assign(&mut walk[r..r + w], &new[r..r + w]); + c.sweep_words += w as u64; + set(&mut next_dirty[l * w..l * w + w], x); + next_rows[l].push(x); + fresh = true; + } + } + } else { + let span = row(l, 0)..row(l, 0) + n * w; + mask_andnot_assign(&mut new[span.clone()], &walk[span.clone()]); + c.sweep_words += 2 * (n * w) as u64; + if mask_any(&new[span.clone()]) { + mask_or_assign(&mut walk[span.clone()], &new[span.clone()]); + c.sweep_words += (n * w) as u64; + fresh = true; + if opt.skip_clean { + // Without a written-row list, the dirty rows need a scan. + for x in 0..n { + let r = row(l, x); + if mask_any(&new[r..r + w]) { + set(&mut next_dirty[l * w..l * w + w], x); + } + } + c.sweep_words += (n * w) as u64; + } + } + } + if fresh { + any = true; + } else { + *is_live = false; + } + } + if opt.touched_only { + for (l, rows) in n_rows.iter_mut().enumerate() { + for &x in rows.iter() { + n_mark[l * w + x / 64] &= !(1 << (x % 64)); + } + rows.clear(); + } + } + if !any { + break; + } + std::mem::swap(&mut delta, &mut new); + if opt.touched_only { + // `new` now holds the old delta: clear only its non-zero rows. + for (l, rows) in d_rows.iter().enumerate() { + for &x in rows { + let r = row(l, x); + new[r..r + w].fill(0); + c.sweep_words += w as u64; + } + } + // Written rows with nothing fresh are zero already (new &= !walk + // left nothing), so the next delta's rows are exactly next_rows. + } + d_dirty = next_dirty; + d_rows = next_rows; + live = new_live; + } + + Dirty { + planes: Planes { + n, + w, + lens, + node, + walk, + row_reads: c.row_reads, + }, + counters: c, + } +} + +// ---- benchmark ------------------------------------------------------------------- + +fn median(mut v: Vec) -> Duration { + v.sort(); + v[v.len() / 2] +} + +fn timed(mut f: impl FnMut() -> T) -> (Duration, T) { + let mut t = Vec::with_capacity(REPS); + let mut out = None; + for _ in 0..REPS { + let s = Instant::now(); + out = Some(black_box(f())); + t.push(s.elapsed()); + } + (median(t), out.expect("REPS > 0")) +} + +fn status_kib(field: &str) -> usize { + std::fs::read_to_string("/proc/self/status") + .expect("Linux /proc") + .lines() + .find_map(|l| l.strip_prefix(field)) + .and_then(|v| v.trim().trim_end_matches("kB").trim().parse().ok()) + .expect("field present") +} + +/// Child mode: one count-only run, resident high-water growth in KiB. +fn child(arm: &str, fixture: &str) { + let (_, e) = fixtures() + .into_iter() + .find(|(n, _)| *n == fixture) + .expect("known fixture"); + std::fs::write("/proc/self/clear_refs", "5").expect("clear_refs"); + let before = status_kib("VmRSS:"); + let opt = VARIANTS + .iter() + .find(|(n, _)| *n == arm) + .map(|(_, o)| *o) + .expect("known variant"); + black_box(dirty_walk(width(&e), &e, opt).planes.counts()); + println!("{}", status_kib("VmHWM:").saturating_sub(before)); +} + +fn resident_kib(arm: &str, fixture: &str) -> usize { + let out = std::process::Command::new(std::env::current_exe().expect("exe")) + .args(["--resident", arm, fixture]) + .output() + .expect("child runs"); + assert!(out.status.success(), "child failed: {out:?}"); + String::from_utf8(out.stdout) + .expect("utf8") + .trim() + .parse() + .expect("a number") +} + +fn ratio(x: Duration, y: Duration) -> String { + format!("{:.2}×", x.as_secs_f64() / y.as_secs_f64()) +} + +/// A chain of `len` nodes (lengths as in `chain`: 0, and 1 every tenth hop) +/// where each node has `fan - 1` extra edges to distinct nodes 2 to 8 ahead, +/// drawn without replacement. Near the tail fewer than `fan - 1` such nodes +/// exist; [`effective_fan`] reports the distinct out-degree actually built. +fn chain_fan(len: i32, fan: usize, seed: u64) -> Vec { + let mut s = seed; + let mut next = move || { + s = s.wrapping_add(0x9E37_79B9_7F4A_7C15); + let mut z = s; + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) + }; + let mut e = Vec::new(); + for i in 0..len - 1 { + let l = u32::from(i % 10 == 9); + e.push((i, i + 1, l)); + let mut ahead: Vec = (i + 2..=(i + 8).min(len - 1)).collect(); + for k in 0..(fan - 1).min(ahead.len()) { + let pick = k + (next() % (ahead.len() - k) as u64) as usize; + ahead.swap(k, pick); + e.push((i, ahead[k], l)); + } + } + e +} + +/// Distinct out-edges per node that has any. +fn effective_fan(e: &[Edge]) -> f64 { + let distinct: std::collections::BTreeSet<(i32, i32)> = + e.iter().map(|&(a, b, _)| (a, b)).collect(); + let sources: std::collections::BTreeSet = e.iter().map(|&(a, _, _)| a).collect(); + distinct.len() as f64 / sources.len() as f64 +} + +/// The chain sweep: A, A′ (both switches) and C over chain length, fan-out +/// and, through the length, the number of foci. +fn sweep() { + println!("# chain sweep (ms, median of {REPS}; C uses {DEFAULT_M} x {DEFAULT_M} microtiles)\n"); + println!( + "| len | fan | distinct fan | foci | A count | A′ count | C count | C/A′ count | A e2e | A′ e2e | C e2e | C/A′ e2e | oracle |" + ); + println!("|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---|"); + let both = VARIANTS[3].1; + for len in [128, 192, 230, 256, 320, 384, 512, 768] { + for fan in [1, 2, 4] { + let e = chain_fan(len, fan, 0xC4A1 + fan as u64); + let n = width(&e); + let (t_a, a) = timed(|| { + let p = mask_walk(n, &e); + black_box(p.counts()); + p + }); + let (t_d, d) = timed(|| { + let d = dirty_walk(n, &e, both); + black_box(d.planes.counts()); + d + }); + // End to end times the fixpoint without the counts, as B's and C's + // `e2e()` do. + let (t_a_fix, _) = timed(|| mask_walk(n, &e)); + let (t_d_fix, _) = timed(|| dirty_walk(n, &e, both)); + let c: Vec<_> = (0..REPS) + .map(|_| sparse_eval(&e, DEFAULT_M, usize::MAX).expect("C answers")) + .collect(); + let t_c = median(c.iter().map(|r| r.e2e_count()).collect()); + let t_c_e2e = median(c.iter().map(|r| r.e2e()).collect()); + let (t_a_dec, at) = timed(|| a.tuples()); + let (t_d_dec, dt) = timed(|| d.planes.tuples()); + agree(&dt, &at).unwrap_or_else(|x| panic!("len {len} fan {fan} A′: {x}")); + let c_last = c.into_iter().last().expect("REPS > 0"); + agree(&c_last.tuples, &at).unwrap_or_else(|x| panic!("len {len} fan {fan} C: {x}")); + let oracle = if len as usize <= ORACLE_MAX_NODES { + agree(&at, &ascent_tuples(&e)).unwrap_or_else(|x| panic!("{x}")); + agree(&at, &crepe_tuples(&e)).unwrap_or_else(|x| panic!("{x}")); + "= ascent = crepe" + } else { + "UNCHECKED" + }; + println!( + "| {len} | {fan} | {:.2} | {} | {} | {} | {} | {} | {} | {} | {} | {} | {oracle} |", + effective_fan(&e), + c_last.pages, + ms(t_a), + ms(t_d), + ms(t_c), + ratio(t_c, t_d), + ms(t_a_fix + t_a_dec), + ms(t_d_fix + t_d_dec), + ms(t_c_e2e), + ratio(t_c_e2e, t_d_fix + t_d_dec), + ); + } + } +} + +fn main() { + let args: Vec = std::env::args().collect(); + if args.get(1).map(String::as_str) == Some("--resident") { + child(&args[2], &args[3]); + return; + } + if args.get(1).map(String::as_str) == Some("--sweep") { + sweep(); + return; + } + println!("# count-only (ms, median of {REPS})\n"); + println!( + "| fixture | nodes | A | A′ off | A′ skip | A′ touched | A′ both | B | C | C/A′both | A′both/A | oracle |" + ); + println!("|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---|"); + let mut work = Vec::new(); + let mut e2e = Vec::new(); + for (name, e) in fixtures() { + let n = width(&e); + let (t_a, a) = timed(|| mask_walk(n, &e)); + let a_counts = a.counts(); + let mut tv = Vec::new(); + let mut cv = Vec::new(); + for (_, opt) in VARIANTS { + let (t, d) = timed(|| { + let d = dirty_walk(n, &e, opt); + black_box(d.planes.counts()); + d + }); + assert_eq!(d.planes.counts(), a_counts, "{name} {opt:?}: counts"); + tv.push(t); + cv.push(d); + } + // Count-only for A is its fixpoint plus the popcounts; `t_a` is the + // fixpoint alone, which end to end uses (B's and C's `e2e()` skip the + // count phase too). + let (t_a_count, _) = timed(|| mask_walk(n, &e).counts()); + let (t_d_fix, _) = timed(|| dirty_walk(n, &e, VARIANTS[3].1)); + let b = (0..REPS) + .map(|_| tile_eval(&e, usize::MAX).expect("B answers")) + .collect::>(); + let c = (0..REPS) + .map(|_| sparse_eval(&e, DEFAULT_M, usize::MAX).expect("C answers")) + .collect::>(); + let t_b = median(b.iter().map(|r| r.e2e_count()).collect()); + let t_c = median(c.iter().map(|r| r.e2e_count()).collect()); + let t_b_e2e = median(b.iter().map(|r| r.e2e()).collect()); + let t_c_e2e = median(c.iter().map(|r| r.e2e()).collect()); + + // Tuples: every A′ variant against A, and the oracles where they run. + let a_tuples = a.tuples(); + for (d, (vn, _)) in cv.iter().zip(VARIANTS) { + agree(&d.planes.tuples(), &a_tuples).unwrap_or_else(|x| panic!("{name} {vn}: {x}")); + } + let c_last = c.into_iter().last().expect("REPS > 0"); + agree(&c_last.tuples, &a_tuples).unwrap_or_else(|x| panic!("{name} C: {x}")); + let nodes = a_tuples + .walk + .iter() + .filter(|w| w.0 == w.1 && w.2 == 0) + .count(); + let oracle = if nodes <= ORACLE_MAX_NODES { + agree(&a_tuples, &ascent_tuples(&e)).unwrap_or_else(|x| panic!("{name}: {x}")); + agree(&a_tuples, &crepe_tuples(&e)).unwrap_or_else(|x| panic!("{name}: {x}")); + "all = ascent = crepe" + } else { + "all equal; no Datalog oracle (UNCHECKED)" + }; + println!( + "| {name} | {nodes} | {} | {} | {} | {} | {} | {} | {} | {} | {} | {oracle} |", + ms(t_a_count), + ms(tv[0]), + ms(tv[1]), + ms(tv[2]), + ms(tv[3]), + ms(t_b), + ms(t_c), + ratio(t_c, tv[3]), + ratio(tv[3], t_a_count), + ); + // End to end: the fixpoint plus decoding the tuples, no counts. + let both = &cv[3]; + let (t_dec, _) = timed(|| both.planes.tuples()); + let (t_a_dec, _) = timed(|| a.tuples()); + e2e.push((name, t_a + t_a_dec, t_d_fix + t_dec, t_b_e2e, t_c_e2e)); + work.push(( + name, + cv.iter().map(|d| d.counters).collect::>(), + a.row_reads, + )); + } + println!("\n# end to end, tuples decoded (ms)\n"); + println!("| fixture | A | A′ both | B | C | C/A′both | C/A |"); + println!("|---|---:|---:|---:|---:|---:|---:|"); + for (name, a, d, b, c) in &e2e { + println!( + "| {name} | {} | {} | {} | {} | {} | {} |", + ms(*a), + ms(*d), + ms(*b), + ms(*c), + ratio(*c, *d), + ratio(*c, *a) + ); + } + println!("\n# work counts per A′ variant (off / skip / touched / both)\n"); + println!( + "| fixture | iterations | A row reads | row reads | bits masked out | words swept | adjacency words scanned |" + ); + println!("|---|---:|---:|---|---|---|---:|"); + for (name, cs, a_reads) in &work { + let col = |f: fn(&Counters) -> u64| { + cs.iter() + .map(|c| f(c).to_string()) + .collect::>() + .join(" / ") + }; + println!( + "| {name} | {} | {a_reads} | {} | {} | {} | {} |", + cs[0].iterations, + col(|c| c.row_reads), + col(|c| c.masked_out), + col(|c| c.sweep_words), + cs[3].adj_words, + ); + } + println!("\n# resident memory, count-only (MiB, VmHWM growth in a fresh process)\n"); + println!("| fixture | A′ off | A′ both |"); + println!("|---|---:|---:|"); + for (name, ..) in &work { + println!( + "| {name} | {} | {} |", + mib(resident_kib("A′ off", name) * 1024), + mib(resident_kib("A′ both", name) * 1024) + ); + } +} + +// ---- tests ----------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use sparse::tile::baseline::tests::random_graph; + use sparse::tile::chain; + use sparse::{clustered, cyclic, dense}; + + /// Every variant's tuples equal ascent's, crepe's and A's. + fn check(e: &[Edge]) -> Vec { + let n = width(e); + let a = mask_walk(n, e); + let at = a.tuples(); + agree(&at, &ascent_tuples(e)).unwrap_or_else(|d| panic!("A ascent: {d}")); + agree(&at, &crepe_tuples(e)).unwrap_or_else(|d| panic!("A crepe: {d}")); + let mut out = Vec::new(); + for (name, opt) in VARIANTS { + let d = dirty_walk(n, e, opt); + assert_eq!(d.planes.counts(), a.counts(), "{name}: counts"); + agree(&d.planes.tuples(), &at).unwrap_or_else(|x| panic!("{name}: {x}")); + out.push(d.counters); + } + // Off is A's loop: the same gathers, one by one. + assert_eq!(out[0].row_reads, a.row_reads, "A′ off is not A's loop"); + out + } + + #[test] + fn mixed_lengths_with_cycles() { + check(&random_graph(65, 2, 0x5EED_0001)); + } + + #[test] + fn alternative_path_lengths() { + check(&[(0, 1, 1), (0, 2, 2), (2, 1, 3), (1, 3, 7), (0, 3, 5)]); + } + + #[test] + fn zero_length_cycle_and_over_length_edges() { + let mut e: Vec = (0..200).map(|i| (i, (i + 1) % 200, 0)).collect(); + e.push((0, 100, 7)); + e.push((5, 6, 51)); + let c = check(&e); + assert!(c[3].iterations >= 150); + } + + #[test] + fn deep_propagation_and_word_boundaries() { + // 300 nodes: rows span five words, ids cross 64-bit boundaries. + let c = check(&chain(300)); + assert!(c[3].iterations >= 200); + } + + #[test] + fn the_shared_fixtures() { + check(&clustered(96, 2)); + check(&dense(48, 10, 3)); + check(&cyclic(80)); + } + + #[test] + fn skip_clean_skips_and_touched_only_sweeps_less() { + // On a chain the frontier is one row per length: the switches must + // each do what they claim, and neither may change the gathers that + // matter. + let c = check(&chain(230)); + let [off, skip, touched, both] = [c[0], c[1], c[2], c[3]]; + assert_eq!(off.masked_out, 0); + assert!(skip.masked_out > 0, "skip_clean masked nothing"); + assert_eq!(skip.row_reads + skip.masked_out, off.row_reads); + assert_eq!(touched.row_reads, off.row_reads); + assert!( + touched.sweep_words * 10 < off.sweep_words, + "touched_only swept {} of {} words", + touched.sweep_words, + off.sweep_words + ); + assert_eq!(both.row_reads, skip.row_reads); + assert!(both.sweep_words <= touched.sweep_words); + } + + #[test] + fn the_sweep_generator_across_foci() { + // Fan-out 4 over two foci, oracle-checked like every sweep point. + let e = chain_fan(300, 4, 0xC4A5); + check(&e); + let r = sparse_eval(&e, DEFAULT_M, usize::MAX).expect("C answers"); + assert_eq!(r.pages, 2); + agree(&r.tuples, &ascent_tuples(&e)).unwrap_or_else(|d| panic!("C: {d}")); + } + + #[test] + fn chain_fan_builds_distinct_successors() { + for fan in [1, 2, 4] { + let e = chain_fan(200, fan, 0xC4A1 + fan as u64); + let distinct: std::collections::BTreeSet<(i32, i32)> = + e.iter().map(|&(a, b, _)| (a, b)).collect(); + assert_eq!(distinct.len(), e.len(), "fan {fan}: duplicate edges"); + // Away from the tail every node has exactly `fan` successors. + for i in 0..190 { + assert_eq!( + e.iter().filter(|x| x.0 == i).count(), + fan, + "fan {fan} node {i}" + ); + } + } + assert!((effective_fan(&chain_fan(200, 1, 1)) - 1.0).abs() < 1e-9); + } + + #[test] + fn a_dense_frontier_gives_skip_clean_nothing_to_skip() { + // Silence twin: when every delta row is dirty, nothing is masked. + let e: Vec = (0..40) + .flat_map(|x| (0..40).map(move |y| (x, y, 1))) + .collect(); + let c = check(&e); + assert_eq!(c[1].masked_out, 0, "masked bits on an all-dirty frontier"); + } +} diff --git a/crates/crepe-plan-probe/examples/walk_mask.rs b/crates/crepe-plan-probe/examples/walk_mask.rs index 231608b56..8a94d28e8 100644 --- a/crates/crepe-plan-probe/examples/walk_mask.rs +++ b/crates/crepe-plan-probe/examples/walk_mask.rs @@ -172,26 +172,26 @@ pub(crate) fn crepe_tuples(e: &[(i32, i32, u32)]) -> Tuples { } /// Words per row of an `n`-wide plane. -fn words(n: usize) -> usize { +pub(crate) fn words(n: usize) -> usize { n.div_ceil(64) } -fn set(row: &mut [u64], i: usize) { +pub(crate) fn set(row: &mut [u64], i: usize) { row[i / 64] |= 1 << (i % 64); } -fn bit(row: &[u64], i: usize) -> bool { +pub(crate) fn bit(row: &[u64], i: usize) -> bool { row[i / 64] >> (i % 64) & 1 == 1 } /// The fixpoint of the program over bit planes. pub(crate) struct Planes { - n: usize, - w: usize, - lens: usize, - node: Vec, + pub(crate) n: usize, + pub(crate) w: usize, + pub(crate) lens: usize, + pub(crate) node: Vec, /// `walk[l][x]`: the nodes `z` with `Walk(x, z, l)`. - walk: Vec, + pub(crate) walk: Vec, /// Value-addressed row reads (`delta[row(l2, y)]` for a set bit `y`). pub(crate) row_reads: u64, } diff --git a/crates/crepe-plan-probe/examples/walk_sparse.rs b/crates/crepe-plan-probe/examples/walk_sparse.rs index 1e94daa63..ccb4a8a95 100644 --- a/crates/crepe-plan-probe/examples/walk_sparse.rs +++ b/crates/crepe-plan-probe/examples/walk_sparse.rs @@ -69,7 +69,7 @@ use mork_pathmap_probe::rmap::{Refusal, SLOTS, TILE_ROWS, Witness}; // Arm B (and, inside it, arm A and the oracles), compiled from its own file. #[allow(dead_code)] #[path = "walk_tile.rs"] -mod tile; +pub(crate) mod tile; use tile::baseline::{Tuples, agree, ascent_tuples, crepe_tuples, edges, mask_walk}; use tile::{ @@ -77,19 +77,19 @@ use tile::{ project, tile_eval, tuples, width, }; -const REPS: usize = 5; -const ORACLE_MAX_NODES: usize = 512; -const ROW_WORDS: usize = SLOTS / 64; +pub(crate) const REPS: usize = 5; +pub(crate) const ORACLE_MAX_NODES: usize = 512; +pub(crate) const ROW_WORDS: usize = SLOTS / 64; /// The microtile side used by the main comparison, fixed before any run. -const DEFAULT_M: usize = 64; -const MICROTILES: [usize; 4] = [16, 32, 64, 256]; +pub(crate) const DEFAULT_M: usize = 64; +pub(crate) const MICROTILES: [usize; 4] = [16, 32, 64, 256]; // ---- prepared metadata --------------------------------------------------------- /// Bit `y` set when row `y` of a tile has any bit. -type RowOcc = [u64; ROW_WORDS]; +pub(crate) type RowOcc = [u64; ROW_WORDS]; -fn row_occ(t: &[u64]) -> RowOcc { +pub(crate) fn row_occ(t: &[u64]) -> RowOcc { let mut o = [0u64; ROW_WORDS]; for y in 0..SLOTS { if t[y * ROW_WORDS..(y + 1) * ROW_WORDS] @@ -103,11 +103,11 @@ fn row_occ(t: &[u64]) -> RowOcc { } /// The edge tiles with their row occupancy, built once. -struct Prepared { - by_len: Vec<(usize, Vec>)>, +pub(crate) struct Prepared { + pub(crate) by_len: Vec<(usize, Vec>)>, } -fn prepare(p: &Projection) -> Prepared { +pub(crate) fn prepare(p: &Projection) -> Prepared { Prepared { by_len: p .by_len @@ -124,7 +124,7 @@ fn prepare(p: &Projection) -> Prepared { /// The `m`-bit segment `[yb * m, (yb + 1) * m)` of a 256-bit row, as /// `(word, shift, len)` pieces of at most one word each. -fn segments(yb: usize, m: usize) -> impl Iterator { +pub(crate) fn segments(yb: usize, m: usize) -> impl Iterator { let lo = yb * m; let hi = lo + m; (lo / 64..hi.div_ceil(64)).map(move |w| { @@ -134,7 +134,7 @@ fn segments(yb: usize, m: usize) -> impl Iterator }) } -fn low(len: usize) -> u64 { +pub(crate) fn low(len: usize) -> u64 { if len == 64 { u64::MAX } else { @@ -144,7 +144,7 @@ fn low(len: usize) -> u64 { /// Occupancy of the `m x m` blocks of a delta tile: cell `zb * nb + yb` set /// when rows `zb * m ..` have a bit in columns `yb * m ..`. -fn block_occ(t: &[u64], m: usize) -> Vec { +pub(crate) fn block_occ(t: &[u64], m: usize) -> Vec { let nb = SLOTS / m; let mut occ = vec![0u64; (nb * nb).div_ceil(64)]; for zb in 0..nb { @@ -164,26 +164,26 @@ fn block_occ(t: &[u64], m: usize) -> Vec { // ---- execution ------------------------------------------------------------------- #[derive(Clone, Copy, Debug, Default)] -struct Counters { +pub(crate) struct Counters { /// Tile triples whose edge and delta tiles both exist. - tile_products: u64, + pub(crate) tile_products: u64, /// Occupied delta blocks visited. - blocks: u64, + pub(crate) blocks: u64, /// Row ORs executed (one 256-bit OR each). - row_ors: u64, + pub(crate) row_ors: u64, /// mask-risc calls (the per-tile update). - calls: u64, + pub(crate) calls: u64, } /// The update programs and their buffers. -struct Exec { - fresh_prog: Program, - or_prog: Program, - scratch: Scratch<'static>, - buf: Box<[u64]>, +pub(crate) struct Exec { + pub(crate) fresh_prog: Program, + pub(crate) or_prog: Program, + pub(crate) scratch: Scratch<'static>, + pub(crate) buf: Box<[u64]>, } -fn update_programs() -> (Program, Program) { +pub(crate) fn update_programs() -> (Program, Program) { let keep = Terminal::Keep { mask: Operand::Scratch(0), }; @@ -208,7 +208,7 @@ fn update_programs() -> (Program, Program) { } impl Exec { - fn with_programs(fresh_prog: Program, or_prog: Program) -> Result { + pub(crate) fn with_programs(fresh_prog: Program, or_prog: Program) -> Result { admit(&fresh_prog)?; admit(&or_prog)?; Ok(Exec { @@ -219,13 +219,19 @@ impl Exec { }) } - fn new() -> Exec { + pub(crate) fn new() -> Exec { let (f, o) = update_programs(); Exec::with_programs(f, o).expect("the update programs have no gather") } /// `out = prog(a, b)` over one tile. - fn run(&mut self, prog_is_fresh: bool, a: &[u64], b: &[u64], n: &mut Counters) -> Box<[u64]> { + pub(crate) fn run( + &mut self, + prog_is_fresh: bool, + a: &[u64], + b: &[u64], + n: &mut Counters, + ) -> Box<[u64]> { let masks: [&[u64]; 2] = [a, b]; let pl = Planes { n_rows: TILE_ROWS, @@ -254,7 +260,7 @@ impl Exec { /// `D(y, z)` (row `z`), `c` is `C(x, z)` (row `z`). Every index is a loop /// coordinate; see the module docs. #[allow(clippy::too_many_arguments)] -fn product_into( +pub(crate) fn product_into( a: &[u64], a_rows: &RowOcc, d: &[u64], @@ -295,12 +301,12 @@ fn product_into( } } -struct Sparse { - run: Run, - counters: Counters, +pub(crate) struct Sparse { + pub(crate) run: Run, + pub(crate) counters: Counters, } -fn recurse( +pub(crate) fn recurse( b: &Bound, p: &Projection, prep: &Prepared, @@ -410,33 +416,33 @@ fn recurse( // ---- the whole arm C ----------------------------------------------------------- -struct Report { - attest: Duration, - bind: Duration, - project: Duration, - prepare: Duration, - recurse: Duration, - count: Duration, - decode: Duration, - iterations: usize, - counters: Counters, - pages: usize, - lookups: u64, - decodes: u64, - counts: (usize, usize), - tuples: Tuples, +pub(crate) struct Report { + pub(crate) attest: Duration, + pub(crate) bind: Duration, + pub(crate) project: Duration, + pub(crate) prepare: Duration, + pub(crate) recurse: Duration, + pub(crate) count: Duration, + pub(crate) decode: Duration, + pub(crate) iterations: usize, + pub(crate) counters: Counters, + pub(crate) pages: usize, + pub(crate) lookups: u64, + pub(crate) decodes: u64, + pub(crate) counts: (usize, usize), + pub(crate) tuples: Tuples, } impl Report { - fn e2e_count(&self) -> Duration { + pub(crate) fn e2e_count(&self) -> Duration { self.attest + self.bind + self.project + self.prepare + self.recurse + self.count } - fn e2e(&self) -> Duration { + pub(crate) fn e2e(&self) -> Duration { self.attest + self.bind + self.project + self.prepare + self.recurse + self.decode } } -fn sparse_eval(e: &[Edge], m: usize, max_pages: usize) -> Result { +pub(crate) fn sparse_eval(e: &[Edge], m: usize, max_pages: usize) -> Result { let s = Instant::now(); let facts = attest(e); let attest_t = s.elapsed(); @@ -480,7 +486,7 @@ fn sparse_eval(e: &[Edge], m: usize, max_pages: usize) -> Result // ---- fixtures ------------------------------------------------------------------ /// splitmix64. -fn rng(seed: u64) -> impl FnMut() -> u64 { +pub(crate) fn rng(seed: u64) -> impl FnMut() -> u64 { let mut s = seed; move || { s = s.wrapping_add(0x9E37_79B9_7F4A_7C15); @@ -492,7 +498,7 @@ fn rng(seed: u64) -> impl FnMut() -> u64 { } /// `deg` random out-edges per node, lengths 1, 2, 3, 5 or 7. -fn sparse(n: i32, deg: usize, seed: u64) -> Vec { +pub(crate) fn sparse(n: i32, deg: usize, seed: u64) -> Vec { let mut r = rng(seed); let lens = [1, 2, 3, 5, 7]; let mut e = Vec::new(); @@ -506,7 +512,7 @@ fn sparse(n: i32, deg: usize, seed: u64) -> Vec { /// Communities of 32 nodes, 4 internal edges per node (lengths 1..=3), and /// one cross edge per community (length 5). -fn clustered(n: i32, seed: u64) -> Vec { +pub(crate) fn clustered(n: i32, seed: u64) -> Vec { let mut r = rng(seed); let mut e = Vec::new(); for x in 0..n { @@ -523,7 +529,7 @@ fn clustered(n: i32, seed: u64) -> Vec { } /// Every ordered pair with probability `pct` %, length 1 or 2. -fn dense(n: i32, pct: u64, seed: u64) -> Vec { +pub(crate) fn dense(n: i32, pct: u64, seed: u64) -> Vec { let mut r = rng(seed); let mut e = Vec::new(); for x in 0..n { @@ -537,14 +543,14 @@ fn dense(n: i32, pct: u64, seed: u64) -> Vec { } /// A zero-length ring, a length-1 ring and chords of length 7. -fn cyclic(n: i32) -> Vec { +pub(crate) fn cyclic(n: i32) -> Vec { let mut e: Vec = (0..n).map(|i| (i, (i + 1) % n, 0)).collect(); e.extend((0..n).map(|i| (i, (i + 2) % n, 1))); e.extend((0..n).step_by(17).map(|i| (i, (i + n / 2) % n, 7))); e } -fn fixtures() -> Vec<(&'static str, Vec)> { +pub(crate) fn fixtures() -> Vec<(&'static str, Vec)> { vec![ ("sparse(512)", sparse(512, 1, 1)), ("clustered(512)", clustered(512, 2)), @@ -560,12 +566,12 @@ fn fixtures() -> Vec<(&'static str, Vec)> { // ---- benchmark ------------------------------------------------------------------- -fn median(mut v: Vec) -> Duration { +pub(crate) fn median(mut v: Vec) -> Duration { v.sort(); v[v.len() / 2] } -fn status_kib(field: &str) -> usize { +pub(crate) fn status_kib(field: &str) -> usize { std::fs::read_to_string("/proc/self/status") .expect("Linux /proc") .lines() @@ -575,7 +581,7 @@ fn status_kib(field: &str) -> usize { } /// Child mode: one arm on one fixture, resident high-water growth in KiB. -fn child(arm: &str, fixture: &str) { +pub(crate) fn child(arm: &str, fixture: &str) { let (_, e) = fixtures() .into_iter() .find(|(n, _)| *n == fixture) @@ -615,7 +621,7 @@ fn child(arm: &str, fixture: &str) { println!("{}", status_kib("VmHWM:").saturating_sub(before)); } -fn resident_kib(arm: &str, fixture: &str) -> usize { +pub(crate) fn resident_kib(arm: &str, fixture: &str) -> usize { let out = std::process::Command::new(std::env::current_exe().expect("exe")) .args(["--resident", arm, fixture]) .output()