feat(moe): register mixed-qtype decode tables for hot-only storage

[maikzz32]

Stacked on #701, which adds MoeHybridColdBackend::None. Please review that one first — the diff against main shown here contains both. The change belonging to this PR is the last commit, 16 lines in deepseek4_loader.cpp.

Adaptive (mixed ROCmFPX) experts keep their codebooks out of band and need a decode table registered per resident tensor. register_deepseek4_moe_hybrid_mix_tables demanded a materialized GPU cold owner, so an owner using cold-owner None — whose resident set is exactly its hot experts — was refused.

That refusal was unnecessary. The registrar already handles this shape: ds4_register_compact_mix_tensor returns success for a null tensor whose expert-id list is empty, which is precisely how an absent cold owner presents itself. Only the guard was too narrow. It now accepts either a materialized GPU cold owner, as before, or hot-only storage with no cold owner; a Cpu cold owner is still refused, and nothing else changes.

This also resolves the TODO(cluster-verify) I left beside ds4_cluster_has_mix_experts in my own tree.

Why it matters

It is what makes an adaptive artifact usable by an owner whose cold set lives outside the process. Measured downstream on two Radeon 8060S nodes under ROCm 10 with DeepSeek V4 Flash, each node holding half the routed experts with cold owner None: the adaptive artifact loads where it previously failed with mixed expert qtypes require materialized GPU owners.

Against the uniform artifact on the same two nodes, verify width 4, output byte-identical to a single node:

artifact	benchmark prompt	free prompt	exact-copy fidelity
ROCMFP2-STRIX (uniform)	38.4 tok/s	17.2 tok/s	12/60
ROCMFPX-MIX-STRIX (adaptive)	42.3 tok/s	20.6 tok/s	60/60

Fidelity is the exact-copy protocol from server/docs/DS4.md — 20 identifiers × 3 repeats at temperature 0. The adaptive artifact is not merely faster here; it is the one that answers correctly.

Verification

Built against this PR's base with -DDFLASH27B_GPU_BACKEND=hip -DDFLASH27B_HIP_ARCHITECTURES=gfx1151 -DDFLASH27B_ROCMFP2_AFFINE=ON -DGGML_HIP_GRAPHS=ON on a gfx1151 host: 379/379, no new warnings.

🤖 Generated with Claude Code

[cubic-dev-ai]

3 issues found across 5 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/src/deepseek4/deepseek4_loader.cpp">

<violation number="1" location="server/src/deepseek4/deepseek4_loader.cpp:1357">
P1: The new `None` branch is unreachable from DeepSeek4 initialization, so hot-only mixed artifacts still never reach this registration path. Wire the hot-only/cluster configuration to set `cold_expert_backend = MoeHybridColdBackend::None` and invoke this registration for that mode; otherwise this change does not enable the advertised external-cold-owner flow.</violation>
</file>

<file name="server/src/common/moe_hybrid_ffn_eval.cpp">

<violation number="1" location="server/src/common/moe_hybrid_ffn_eval.cpp:4292">
P3: `eval_moe_shared_expert_batched` is exported here but has no call site anywhere in the repository, so this cluster shared-expert implementation never runs. Wire it into the cluster all-reduce path or remove it and its declaration.</violation>

<violation number="2" location="server/src/common/moe_hybrid_ffn_eval.cpp:4302">
P2: When `n_tokens` is negative, this cast occurs before the `n_tokens <= 0` guard, causing a huge allocation request. Check `n_tokens` before converting it to `size_t`.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: The new None branch is unreachable from DeepSeek4 initialization, so hot-only mixed artifacts still never reach this registration path. Wire the hot-only/cluster configuration to set cold_expert_backend = MoeHybridColdBackend::None and invoke this registration for that mode; otherwise this change does not enable the advertised external-cold-owner flow.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/deepseek4/deepseek4_loader.cpp, line 1357:

<comment>The new `None` branch is unreachable from DeepSeek4 initialization, so hot-only mixed artifacts still never reach this registration path. Wire the hot-only/cluster configuration to set `cold_expert_backend = MoeHybridColdBackend::None` and invoke this registration for that mode; otherwise this change does not enable the advertised external-cold-owner flow.</comment>

<file context>
@@ -1346,11 +1346,24 @@ bool register_deepseek4_moe_hybrid_mix_tables(
+    // precisely how an absent cold owner presents itself. Only this check stood
+    // in the way.
+    const bool hot_only =
+        storage.cold_backend_kind == MoeHybridColdBackend::None &&
+        !storage.materialized_cold_experts;
+    const bool gpu_owners =
</file context>

[cubic-dev-ai]

P2: When n_tokens is negative, this cast occurs before the n_tokens <= 0 guard, causing a huge allocation request. Check n_tokens before converting it to size_t.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/common/moe_hybrid_ffn_eval.cpp, line 4302:

<comment>When `n_tokens` is negative, this cast occurs before the `n_tokens <= 0` guard, causing a huge allocation request. Check `n_tokens` before converting it to `size_t`.</comment>

<file context>
@@ -4232,4 +4283,73 @@ bool eval_moe_hybrid_ffn_gpu_resident(
+    std::vector<float> &            out,
+    std::string *                   err) {
+    const int n_embd = cfg.n_embd;
+    out.assign((size_t)n_embd * (size_t)n_tokens, 0.0f);
+    if (n_tokens <= 0) return true;
+    if (!desc.ffn_up_shexp || !desc.ffn_gate_shexp || !desc.ffn_down_shexp) {
</file context>

Suggested change

	out.assign((size_t)n_embd * (size_t)n_tokens, 0.0f);
	if (n_tokens <= 0) {
	out.clear();
	return true;
	}
	out.assign((size_t)n_embd * (size_t)n_tokens, 0.0f);

[cubic-dev-ai]

P3: eval_moe_shared_expert_batched is exported here but has no call site anywhere in the repository, so this cluster shared-expert implementation never runs. Wire it into the cluster all-reduce path or remove it and its declaration.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/common/moe_hybrid_ffn_eval.cpp, line 4292:

<comment>`eval_moe_shared_expert_batched` is exported here but has no call site anywhere in the repository, so this cluster shared-expert implementation never runs. Wire it into the cluster all-reduce path or remove it and its declaration.</comment>

<file context>
@@ -4232,4 +4283,73 @@ bool eval_moe_hybrid_ffn_gpu_resident(
+// cleared), all-reduces it, and adds this locally computed term afterwards.
+// The graph is cached per n_tokens in storage.shared_batched_graph, which
+// release_graph_caches() already frees.
+bool eval_moe_shared_expert_batched(
+    ggml_backend_t                  gpu_backend,
+    const MoeHybridConfig &         cfg,
</file context>