wire: replace 4 MiB script slab with tiered arena allocator#2565
Draft
Roasbeef wants to merge 7 commits into
Draft
wire: replace 4 MiB script slab with tiered arena allocator#2565Roasbeef wants to merge 7 commits into
Roasbeef wants to merge 7 commits into
Conversation
In this commit, we add a small bump allocator (a script arena) intended to replace the fixed 4 MiB script slab used to stage scripts and witness items during transaction decode. The slab approach suffers from memory amplification: decoding even a tiny transaction pins a full 4 MiB buffer, a pool miss allocates and zeroes 4 MiB, and the channel-based free list can retain up to 125 slabs (~524 MB) indefinitely since channels are invisible to GC pressure. The arena instead draws fixed-size chunks from a ladder of size classes (16 KiB, 128 KiB, 1 MiB, 4 MiB) backed by sync.Pool, and bump-allocates scripts out of them. A decode only touches memory proportional to the scripts it actually contains: typical transactions are served entirely by a single 16 KiB chunk, while full blocks walk up to the larger classes. Because the backing pools are sync.Pools, idle chunks are released back to the runtime across GC cycles rather than being pinned forever. Safety falls out of the existing decode structure: staged scripts are copied into a single exactly-sized allocation before a decode returns, so no arena memory ever escapes a decode call and reuse needs no reference counting. The arena caps total staged bytes at the same 4 MiB the slab enforced, preserving the existing DoS bounds.
In this commit, we swap the 4 MiB script slab for the script arena in the transaction and block decode paths. MsgTx.BtcDecode now borrows an arena starting at the 16 KiB class, while MsgBlock.BtcDecode and DeserializeTxLoc start at the 1 MiB class and share one arena across all transactions in the block. Each btcDecode call rewinds the arena on entry, which mirrors the old behavior of handing each transaction the slab from offset zero: the previous transaction's scripts have already been copied into their final exactly-sized buffer by then. ReadTxOut previously stack-allocated an entire scriptSlab per call, and worse, returned a PkScript aliasing that 4 MiB buffer, pinning it for the lifetime of the TxOut. It now stages via the arena and copies the script into an exactly-sized allocation the caller owns. Benchmarks on an M4 Max: ReadTxOut drops from ~55 us and 4.19 MB/op to ~71 ns and 80 B/op, DeserializeTxSmall and DeserializeBlock are at parity, and ReadTxIn picks up ~3 ns from the per-iteration rewind. DeserializeTxLarge shows +12% time and two extra allocs in the benchmark harness only: its tiny heap GCs every few iterations, which drains the sync.Pool and forces chunk re-allocation, the flip side of the pools actually giving memory back.
In this commit, we tighten the script arena against caller misuse and
stabilize large chunk reuse. A released arena is now marked dead:
release is idempotent (a double release no longer double-inserts the
arena into the pool where two decodes could receive it), rewinding a
dead arena is a no-op, and any alloc through a stale reference fails
with errScriptArenaFull instead of silently carving memory out of
chunks another decode may now own. Forgetting release remains safe by
construction, the chunks are ordinary GC-visible slices, so an
abandoned arena is simply collected and only the pooling benefit is
lost.
We also front the two large chunk classes with small bounded free
lists (16x 1 MiB, 8x 4 MiB) layered over the sync.Pool. Large chunks
are the expensive ones to re-create (a pool miss zeroes the whole
chunk) and only a handful are ever live at once, so pinning a few buys
deterministic block decode performance for at most 48 MiB, and only
once block traffic has warmed them. The sync.Pool absorbs bursts past
the fixed lists and lets the GC reclaim the overflow. The hot rewind
and alloc paths are also split so their common cases inline.
benchstat vs master (M4 Max, count=10):
DeserializeTxSmallParallel 382.4n ± 1% 334.6n ± 1% -12.52%
DeserializeBlockParallel 234.7µ ± 2% 248.0µ ± 9% +5.70%
ReadTxOut 29.25µ ± 2% 73.19n ± 1% -99.75%
ReadTxIn 25.94n ± 1% 25.58n ± 1% -1.39%
DeserializeTxSmall 176.6n ± 1% 177.1n ± 1% ~
DeserializeTxLarge 205.0µ ± 1% 216.0µ ± 1% +5.33%
DeserializeBlock 903.6µ ± 2% 926.5µ ± 0% +2.53%
The parallel small transaction case (many peers relaying mempool txns
at once) is the dominant real-world decode load, and it improves 12.5%
on top of the 256x reduction in transient memory per decode. The
DeserializeTxLarge deltas come from the benchmark's tiny heap GCing
every few iterations and draining the small-class sync.Pools, the flip
side of those pools actually returning memory.
In this commit, we put the arena through property-based testing with pgregory.net/rapid. Three properties are checked over randomly generated transactions and blocks: serialize/deserialize is the identity, the raw allocator matches a simple capacity model with no aliasing between live allocations, and decoded messages own all of their memory. The ownership property is checked by poisoning the arena chunk pools after a decode completes and verifying the decoded message still re-serializes to the original bytes, which would fail if any script still aliased arena memory. The generators bias script sizes small like real traffic but periodically exceed the 16 KiB starting chunk class so the growth ladder is exercised, and witnessy transactions are generated with the same shape the decoder produces. A separate unit test locks in the release misuse guards: double release is a no-op and a released arena refuses to allocate, even after a rewind.
In this commit, we tidy the root module after the wire module picked up pgregory.net/rapid v1.3.0 for its property-based tests. The root module replaces wire/v2 with the local directory, so wire's requirements flow into the root module graph and the root's existing rapid requirement moves from v1.2.0 to v1.3.0 to match.
|
Ran the benchmarks here. To reduce variance I ran a stress test with N cores in a busy loop until discover the final frequency after the CPU heats up, then used |
In this commit, we sweep through the findings from review of the script arena change. On the allocator side, we add a compile-time assertion that the largest chunk class can satisfy any allocation passing the arena capacity check (grow relies on that coupling), poison a released arena's capacity one past the limit so even zero-length allocs through a stale reference are rejected, and make putScriptChunk panic on a chunk whose size matches no class rather than silently dropping what would have to be internal corruption. We also refresh the comments in btcDecode that still described the old slab free list mechanics, verify with a new test that ReadTxOut's returned script survives the arena being recycled and poisoned, and swap the per-byte require.Equal marker verification in the arena tests for bytes.Count, which drops the wire test suite under -race from ~225s to ~30s. The rapid dependency also moves to the direct require block where it belongs.
In this commit, we add FuzzTxDecode and FuzzBlockDecode, seeded with the existing test vectors. Both decoders parse untrusted wire data with the script arena as the staging layer, so beyond not crashing, the targets check that any input which decodes successfully round-trips: re-serializing and re-decoding must be a fixed point, which would fail if a script aliased recycled arena memory or if the allocator handed out overlapping buffers. Initial runs cover ~1.7M tx execs and ~4.3M block execs without findings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In this PR, we rework how the wire package stages script and witness data during message decode. Today every
MsgTx.BtcDecodeborrows a fixed 4 MiBscriptSlabfrom a channel based free list of up to 125 slabs. That layout amplifies memory usage for the most common traffic on the network: decoding a 300 byte transaction pins a full 4 MiB buffer for the duration of the decode, a pool miss allocates and zeroes 4 MiB, and the channel can retain up to ~524 MB forever, since channels are invisible to the GC's pressure heuristics. The exportedReadTxOutwas worse still: it stack-allocated a fresh slab per call and returned aPkScriptaliasing it, pinning 4 MiB for the lifetime of the output.We replace the slab with a small bump allocator (a script arena) that draws fixed-size chunks from a ladder of size classes: 16 KiB, 128 KiB, 1 MiB, and 4 MiB. A decode only touches memory proportional to the scripts it actually contains: typical transactions are served entirely by a single 16 KiB chunk (a 256x reduction in transient memory per decode), while full blocks start at the 1 MiB class and grow on demand. Total staged bytes per transaction stay capped at the same 4 MiB the slab enforced, so the existing DoS bounds are unchanged.
Chunk pooling
Each size class is backed by a
sync.Pool, so idle chunks are handed back to the runtime across GC cycles instead of being pinned indefinitely. The two large classes are additionally fronted by small bounded free lists (16x 1 MiB, 8x 4 MiB): large chunks are the expensive ones to re-create (a pool miss zeroes the whole chunk) and only a handful are ever live at once, so pinning a few buys deterministic block decode performance for at most 48 MiB, and only once block traffic has actually warmed them. Thesync.Poolabsorbs bursts past the fixed lists and lets the GC reclaim the overflow.Safety
The arena needs no reference counting because no arena memory ever escapes a decode call: every decode already copies its staged scripts into a single exactly-sized allocation before returning, and
ReadTxOutnow does the same. On top of that structural guarantee, the arena defends against misuse. Forgettingreleaseis safe by construction (chunks are ordinary GC visible slices, so an abandoned arena is simply collected), a doublereleaseis an idempotent no-op rather than double-inserting the arena into the pool, and a released arena is poisoned dead: any alloc through a stale reference fails with a decode error instead of silently carving memory out of chunks another decode may now own.Benchmarks
benchstat vs master, Apple M4 Max, count=10:
The parallel small transaction case models many peers relaying mempool transactions at once, which is the dominant real world decode load: it improves 12.5% on top of the transient memory reduction, since the old design serialized every decode through a single channel while
sync.Poolis per-P. B/op and allocs/op are at exact parity everywhere exceptReadTxOut, which drops from 4.19 MB/op to 80 B/op. TheDeserializeTxLargedeltas come from the benchmark's tiny heap GCing every few iterations and draining the small-class pools, the flip side of those pools actually returning memory; a long-lived node heap GCs orders of magnitude less often.Testing
Beyond the existing wire test suite (which passes unchanged under
-race), we add property-based tests usingpgregory.net/rapid. Three properties are checked over randomly generated transactions and blocks: serialize/deserialize is the identity, the raw allocator matches a simple capacity model with no aliasing between live allocations, and decoded messages own all of their memory. The ownership property is checked by poisoning the arena chunk pools after a decode completes and verifying the decoded message still re-serializes to the original bytes, which would fail if any script still aliased arena memory.See each commit message for a detailed description w.r.t the incremental changes.