feat(kvflash): snapshot ledger + QK-pool restore-seed + consume-restored-KV

[dusterbloom]

Wave-1 KVFlash deep-context — the warm-prefill payoff.

New commits: snapshot ledger + QK-pool restore-seed (f457afb, lib+tests); consume-restored-KV on pooled restore instead of re-prefill (cfbf8ed, env KVFLASH_RESTORE_CONSUME, default-on); drafter-history reconstruction from ledger (14b295e).

Stacks on: #445 (cache-persistence), then #430 core, #429 pager, #428 reservation. Lower commits in this diff are those predecessors; review/merge in order — this rebases to drop them as they land.

Reviewer caveat (correctness): KVFLASH_RESTORE_CONSUME=1 output diverges from the no-consume AR path between ~35K and ~60K tokens. Root cause: chunk-to-block placement permutation changes the FlashAttention key-reduction order, FP non-associativity perturbs the logits, and a near-tie greedy argmax flips. See bench/abc_cache_harness/phase3_divergence_rootcause.md. It is an accepted reuse-vs-recompute flip, NOT a correctness bug — but output is NOT bit-identical at long ctx and there is NO agentic gate yet.

The ~0.1s warm-turn speedup is synthetic NIAH single-needle, not a published result. A real-CLI multi-turn output-equivalence capture is the open follow-up before defaulting this on for any widely-read claim.

[cubic-dev-ai]

12 issues found across 24 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="bench/abc_cache_harness/phase3_gate_intraproc.py">

<violation number="1" location="bench/abc_cache_harness/phase3_gate_intraproc.py:199">
P2: Prefill metric extraction conflates absent prefill_s with 0.0, producing misleading speedup/output when parse_done fails.</violation>

<violation number="2" location="bench/abc_cache_harness/phase3_gate_intraproc.py:220">
P2: Gate result can falsely pass when consume=1 is internally non-deterministic because c1_self is not included in pass/fail conditions.</violation>
</file>

<file name="server/src/qwen35moe/qwen35moe_backend.h">

<violation number="1" location="server/src/qwen35moe/qwen35moe_backend.h:111">
P3: Unused member introduces dead state and misleading API surface. Remove it until graph-cache wiring is implemented.</violation>
</file>

<file name="bench/bitplane_lsh_experiment.py">

<violation number="1" location="bench/bitplane_lsh_experiment.py:286">
P2: `k_extra == 0` incorrectly keeps all remaining tokens, invalidating the sink+recent mass-recall numbers at small k. Return an empty extra set when no extra budget is available.</violation>

<violation number="2" location="bench/bitplane_lsh_experiment.py:335">
P2: Script introduces an undeclared runtime dependency on SciPy. This makes the experiment fail in standard project environments unless SciPy is installed out-of-band.</violation>

<violation number="3" location="bench/bitplane_lsh_experiment.py:386">
P2: Default output directory is machine-specific and non-portable. Use a repo-relative default path.</violation>
</file>

<file name="bench/qwen35moe_dflash/ctxsweep/tq3_fast_attention_prior_art.md">

<violation number="1" location="bench/qwen35moe_dflash/ctxsweep/tq3_fast_attention_prior_art.md:32">
P2: Incomplete list of existing vec_dot FA functions — document claims only f16, q4_0, and bf16 exist, but ggml-cuda fattn-common.cuh also has q4_1, q5_0, q5_1, and q8_0. Missing these 4 real types undermines the claim that the vec_dot set is small and makes the selective-instantiation recommendation less grounded.</violation>

<violation number="2" location="bench/qwen35moe_dflash/ctxsweep/tq3_fast_attention_prior_art.md:122">
P1: Wrong claim that vLLM has no sub-8-bit KV support (❌ in table). vLLM merged TurboQuant (PR #38479) on 2026-04-15 — 2+ months before this document's research date — with presets `tq-k3v4nc` (3-bit MSE keys) and `tq-t3nc` (3-bit keys + values). This invalidates the 'floor at 8-bit' narrative.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: Wrong claim that vLLM has no sub-8-bit KV support (❌ in table). vLLM merged TurboQuant (PR #38479) on 2026-04-15 — 2+ months before this document's research date — with presets tq-k3v4nc (3-bit MSE keys) and tq-t3nc (3-bit keys + values). This invalidates the 'floor at 8-bit' narrative.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At bench/qwen35moe_dflash/ctxsweep/tq3_fast_attention_prior_art.md, line 122:

<comment>Wrong claim that vLLM has no sub-8-bit KV support (❌ in table). vLLM merged TurboQuant (PR #38479) on 2026-04-15 — 2+ months before this document's research date — with presets `tq-k3v4nc` (3-bit MSE keys) and `tq-t3nc` (3-bit keys + values). This invalidates the 'floor at 8-bit' narrative.</comment>

<file context>
@@ -0,0 +1,179 @@
+|---|---|---|---|
+| FP8 (E4M3/E5M2) KV | ✅ production (FA3 now runs attention *in* FP8) | ✅ production | ✅ production |
+| INT8 KV | ❌ open RFC (#33480/#37319) | limited | ✅ |
+| 4-bit / 3-bit / 2-bit KV | ❌ | ❌ | ❌ |
+
+- Industry rationale: "quantizing 16-bit KV to 4-bit or 2-bit … quality issues still remain, making it challenging for practical deployment." (SqueezeBits vLLM-vs-TRT blog.)
</file context>

[cubic-dev-ai]

P2: Default output directory is machine-specific and non-portable. Use a repo-relative default path.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At bench/bitplane_lsh_experiment.py, line 386:

<comment>Default output directory is machine-specific and non-portable. Use a repo-relative default path.</comment>

<file context>
@@ -0,0 +1,392 @@
+    parser.add_argument("--k", default="/tmp/phase0_dump/phase0_K.bin")
+    parser.add_argument("--q", default="/tmp/phase0_dump/phase0_Q.bin")
+    parser.add_argument("--out-dir",
+                        default="/home/peppi/Dev/lucebox-hub/bench/qwen35moe_dflash/ctxsweep/")
+    args = parser.parse_args()
+    run_experiment(args.k, args.q, args.out_dir)
</file context>

[cubic-dev-ai]

P2: Script introduces an undeclared runtime dependency on SciPy. This makes the experiment fail in standard project environments unless SciPy is installed out-of-band.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At bench/bitplane_lsh_experiment.py, line 335:

<comment>Script introduces an undeclared runtime dependency on SciPy. This makes the experiment fail in standard project environments unless SciPy is installed out-of-band.</comment>

<file context>
@@ -0,0 +1,392 @@
+            break
+
+    # Spearman rank correlation
+    from scipy.stats import spearmanr
+    rho_1bit, _ = spearmanr(s_true, s_1bit)
+    rho_2bit, _ = spearmanr(s_true, s_2bit)
</file context>

[cubic-dev-ai]

P2: k_extra == 0 incorrectly keeps all remaining tokens, invalidating the sink+recent mass-recall numbers at small k. Return an empty extra set when no extra budget is available.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At bench/bitplane_lsh_experiment.py, line 286:

<comment>`k_extra == 0` incorrectly keeps all remaining tokens, invalidating the sink+recent mass-recall numbers at small k. Return an empty extra set when no extra budget is available.</comment>

<file context>
@@ -0,0 +1,392 @@
+            k_extra = max(0, k_total - n_always)
+            # Top k_extra from remaining by approx
+            rem_scores = scores[remaining]
+            if k_extra > 0 and len(rem_scores) > k_extra:
+                rem_top_idx = np.where(remaining)[0][
+                    np.argpartition(rem_scores, -k_extra)[-k_extra:]
</file context>

Suggested change

	if k_extra > 0 and len(rem_scores) > k_extra:
	rem_top_idx = np.where(remaining)[0][
	np.argpartition(rem_scores, -k_extra)[-k_extra:]
	]
	else:
	rem_top_idx = np.where(remaining)[0]
	if k_extra <= 0:
	rem_top_idx = np.array([], dtype=np.int64)
	elif len(rem_scores) > k_extra:
	rem_top_idx = np.where(remaining)[0][
	np.argpartition(rem_scores, -k_extra)[-k_extra:]
	]
	else:
	rem_top_idx = np.where(remaining)[0]

[cubic-dev-ai]

P2: Incomplete list of existing vec_dot FA functions — document claims only f16, q4_0, and bf16 exist, but ggml-cuda fattn-common.cuh also has q4_1, q5_0, q5_1, and q8_0. Missing these 4 real types undermines the claim that the vec_dot set is small and makes the selective-instantiation recommendation less grounded.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At bench/qwen35moe_dflash/ctxsweep/tq3_fast_attention_prior_art.md, line 32:

<comment>Incomplete list of existing vec_dot FA functions — document claims only f16, q4_0, and bf16 exist, but ggml-cuda fattn-common.cuh also has q4_1, q5_0, q5_1, and q8_0. Missing these 4 real types undermines the claim that the vec_dot set is small and makes the selective-instantiation recommendation less grounded.</comment>

<file context>
@@ -0,0 +1,179 @@
+Per-type vec_dot functions that exist today:
+- `vec_dot_fattn_vec_KQ_f16` (FP16)
+- `vec_dot_fattn_vec_KQ_q4_0` — "Dequantizes and dots using `dp4a`"
+- `vec_dot_fattn_vec_KQ_bf16`
+
+**Why quantized KV cannot use tensor cores:** "tensor cores require dense, regularly-formatted operands. Quantized formats store multiple values packed per element with scale/zero-point metadata. The MMA hardware cannot directly operate on these compressed representations — dequantization must occur first, eliminating the memory efficiency gains." (DeepWiki).
</file context>

[cubic-dev-ai]

P3: Unused member introduces dead state and misleading API surface. Remove it until graph-cache wiring is implemented.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/qwen35moe/qwen35moe_backend.h, line 111:

<comment>Unused member introduces dead state and misleading API surface. Remove it until graph-cache wiring is implemented.</comment>

<file context>
@@ -83,13 +96,20 @@ class Qwen35MoeBackend : public Qwen35Backend {
 
     // Persistent pipelined state (initialized once, reused across requests)
     std::unique_ptr<struct PipelinedDecodeState> pipe_state_;
+    std::unique_ptr<HybridSpecGraphCache> hybrid_spec_graph_cache_;
+    bool spec_microbench_done_ = false;
     bool ensure_pipe_state(int kv_start);
</file context>

[cubic-dev-ai]

1 issue found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/src/qwen35/qwen35_backend.cpp">

<violation number="1" location="server/src/qwen35/qwen35_backend.cpp:1511">
P1: Skipping repool when `has(c)` is true can preserve stale pooled-K vectors across requests. That makes residency scoring use old chunk content and can evict/retain wrong chunks after warm restore.</violation>
</file>

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: Skipping repool when has(c) is true can preserve stale pooled-K vectors across requests. That makes residency scoring use old chunk content and can evict/retain wrong chunks after warm restore.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/qwen35/qwen35_backend.cpp, line 1511:

<comment>Skipping repool when `has(c)` is true can preserve stale pooled-K vectors across requests. That makes residency scoring use old chunk content and can evict/retain wrong chunks after warm restore.</comment>

<file context>
@@ -1488,6 +1508,7 @@ void Qwen35Backend::kvflash_qk_pool_to(int committed) {
     const int ct = kvflash_pager_.chunk_tokens();
     const int sealed = committed / ct;
     for (int c = kvflash_qk_pooled_upto_; c < sealed; c++) {
+        if (kvflash_qk_pool_.has(c)) continue;  // keep restored/host-pooled keys
         const int blk = kvflash_pager_.block_of(c);
         if (blk < 0 || !kvflash_qk_pool_.pool_chunk(cache_.attn_k, blk, ct, c)) {
</file context>