Repository navigation
perf(core): hot core library through one chunked-seq boundary and the Zig reduce (ADR-0205) - #80
Merged
Merged
Conversation
clj's PersistentVector$ChunkedSeq is IChunkedSeq: (map f v) realizes the 32-element leaf holding the cursor per lazy step. cljw's vector seq (an .array_seq view) answered chunked-seq? false, so map/filter/keep over a vector built one lazy node and one interpreted thunk per element: 46x bb on (reduce + (map inc v)), and (first (map f v)) ran f once where clj runs it 32 times. runtime/collection/chunked_seq.zig is the one IChunkedSeq boundary (isChunked, chunkCount, chunkNth, chunkRest) over .chunked_cons and the vector-backed view, with clj's leaf geometry. chunked-seq?, __chunk-count, __chunk-nth, chunk-rest and the O-032 drains read it. subvec and array seqs stay unchunked, as in clj. 8000-element vector, per call, ReleaseSafe -Dwasm (bb in brackets): (reduce + (map inc v)) 19271 -> 1204 us [420] (count (filter even? v)) 19646 -> 927 us [221] (into [] (map inc v)) 18082 -> 1306 us [302] Card: CLJW-CORE-HOT-ZIG Smell-audited: 2: new Layer-0 boundary module, ADR-0205 recorded in memory
ADR-0205 (memory 20261022143844-5e8ab771): one chunked-seq boundary and one reduce kernel behind every eager consumer, not per-consumer loops (the devil's-advocate fork's finished form, adopted). - 2- and 3-coll map keep clj's unchunked shape, one f call per realized node in the same order, but the step body (seq every source, f on the firsts, cons onto (self f rests)) is cljw.internal/__map-step in Zig. The lazy node and the recursion stay in core.clj (-map-n-lazy). O-062. - mapv's single-coll form builds through a transient over the Zig reduce, clj's own shape (it used persistent conj). - O-061 / O-062 ledger rows; D-588 records the missing chunk-first / chunk-next (a pre-existing gap the DA fork surfaced). Per call, ReleaseSafe -Dwasm, increment 1 -> 2 (bb in brackets): (reduce + (map * xs xs)), 1000 doubles 2204 -> 1503 us [245] (mapv inc v), 8000 elements 8203 -> 3578 us [482] Pins measured on bb: multi-coll realization counts (1 after first, 70 after reduce, shortest source wins), nil source, exceptions. Correct under CLJW_GC_TORTURE=1. Card: CLJW-CORE-HOT-ZIG Smell-audited: 2: new reentrant primitive with a manual EvalFrame, ADR-0205 in memory
…VM order The vector seq is chunked (ADR-0205), so the golden's realisation order is the JVM/bb chunked order. placement.yaml regenerated for the private helper. Card: CLJW-CORE-HOT-ZIG Smell-audited: 1: visibility fix, golden updated to the reference order
# Conflicts: # .dev/debt.yaml # .dev/optimizations.md
Each HOF recursion level (chunked map draining a user fn) re-enters eval natively, so every native frame on that path is paid per level of user recursion against the ADR-0195 6 MiB budget. - tree_walk: fn locals, call args and their locs come from a per-thread scratch (tw_scratch, watermark discipline like VmArena) instead of 256-slot stack arrays. - vm: try/catch handler windows live in VmArena; cold opcodes run in an out-of-line stepCold copy so their scratch is not in eval's frame. - gc_trigger: collectIfSafe noinline (it was inlined into the op loop). - serialize v14: fn methods carry frame_slots. Deserialized (bootstrap cache) methods fell back to the full MAX_LOCALS window per call. HOF depth 300 -> 495 (12.7 KB/level); more shrinking follows. Smell-audited: 2: TLS scratch mirrors VmArena discipline; serialize grew 5 lines for a format fix
setErrorFmt and raiseArityNotMatched were inlined into vm.eval / treeWalkCall / callMethodImpl, so their formatter temporaries and the 256-byte arity buffer sat in frames paid per native eval re-entry (ADR-0195 budget). Both are error-only paths; mark them noinline. treeWalkCall 768 -> 344 B, callMethodImpl 656 -> 232 B. HOF recursion through chunked map: 495 -> 610 levels (vm, ReleaseSafe); the legit_hof_recursion e2e (500) passes again. Smell-audited: 1: two noinline annotations on cold error paths
Smell-audited: 0: formatter output only
data/placement.yaml regenerated against the merged runtime (1461 vars). Golden 36/36, phase15_watch_recursion legit_hof_recursion (500) passes, smoke green. Depth 1000 (JVM parity) moves to CLJW-VM-CALL-PATH. Smell-audited: 0: generated index only
The 8.5 MB burst landed in pages earlier suites had already committed on #80's macOS gate (base 208 MB, grown +0.8 MB), so 'the burst grew the committed heap' failed for reasons unrelated to the page heap. A 34 MB burst with a 16 MB margin measures the page heap again. Suite smoke green in the shared image (847 tests); 3/3 in a fresh process. Card: CLJW-GC-HEAP-TEST-FRAGILE (20261022192250-76e31716). Smell-audited: 1: test fragility fixed at its cause, knowledge 20261022192219-564f7893
data/placement.yaml regenerated against the merged runtime. Smell-audited: 0: generated index only
tree_walk.zig: #86 split callMethodImpl/callMethodBody with a 2 KB uninitScratch locals buffer on the native stack; this branch moved the window into the threadlocal tw_scratch for recursion depth. Kept one mechanism: callMethodImpl is a plain call and callMethodBody borrows an exact-size tw_scratch window, which neither 0xaa-fills nor costs native stack. O-080 ledger row amended. data/placement.yaml regenerated. Smell-audited: 2: duplicate per-call buffer removed at merge, ledger amended
BuddhiLW
added a commit
that referenced
this pull request
Oct 23, 2026
The 8.5 MB burst landed in pages earlier suites had already committed on #80's macOS gate (base 208 MB, grown +0.8 MB), so 'the burst grew the committed heap' failed for reasons unrelated to the page heap. A 34 MB burst with a 16 MB margin measures the page heap again. Suite smoke green in the shared image (847 tests); 3/3 in a fresh process. Card: CLJW-GC-HEAP-TEST-FRAGILE (20261022192250-76e31716). Smell-audited: 1: test fragility fixed at its cause, knowledge 20261022192219-564f7893
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
perf(core): hot core library through one chunked-seq boundary and the Zig reduce (ADR-0205)
Step 2 of "make cljw faster than bb" (card CLJW-CORE-HOT-ZIG, decision 20261022012219-31249feb). ADR-0205 is memory 20261022143844-5e8ab771. It includes the devil's-advocate fork, whose finished form (one shared mechanism, no per-consumer loops) was adopted.
What changed
PersistentVector$ChunkedSeqis.runtime/collection/chunked_seq.zigis the one boundary thatchunked-seq?,__chunk-count,__chunk-nth,chunk-restand the O-032 drains read. This also fixes a laziness divergence:(first (map f v))ran f once, and now runs it 32 times as clj does. subvec and array seqs stay unchunked, as in clj. (O-061)mapkeep clj's unchunked one-f-per-node shape. The step body now runs in Zig (cljw.internal/__map-step), while the lazy node stays in core.clj. (O-062)mapvbuilds through a transient over the Zigreduce.chunk-first/chunk-next(pre-existing gap).Numbers
ReleaseSafe -Dwasm, VM backend, loaded host, microseconds per call. v is an 8000-element vector, xs is 1000 doubles.
(reduce + (map inc v))(count (filter even? v))(into [] (map inc v))(mapv inc v)(reduce + (map * xs xs))(transduce (map inc) + 0 v)(reduce + v)What remains is the per-element user-fn call through
invokeCallableandvm.eval, plus the scratch-buffer memset that step 1 targets (18-44% of samples). That is a VM call-path change, not more core.clj-to-Zig moves.Tests
test/clj/suites/chunked_seq_test.clj: realization counts 32 and 27,chunk-restcounts, multi-coll realization order and the shortest-source rule.chunked_seq.zigunit tests.CLJW_GC_TORTURE=1.