feat(sglang): account DeepSeek-V4 runtime reservations

[shipiyouniao]

Summary

This is a partial first step for DeepSeek-V4 support. The broader DeepSeek-V4 tracking issue should remain open for the remaining pooling work.

This PR adds a partial SGLang DeepSeek-V4 integration step: kvcached now accounts for DeepSeek-V4 runtime-owned KV/cache pools as static reservations, without taking ownership of those pools.

It also includes a small SGLang allocator compatibility fix so the elastic paged allocator accepts a precomputed num_new_pages argument when the runtime provides it.

What this PR does

• Register per-pool runtime-owned DeepSeek-V4 reservations from SGLang's DeepSeekV4TokenToKVPool.
• Subtract runtime-owned reservations from kvcached's available-device-memory budget before allocating kvcached-managed KV tensors.
• Preserve per-pool reservation breakdown for observability/debugging.
• Accept precomputed num_new_pages in the SGLang paged allocator path.
• Handle newer SGLang allocator layouts where paged allocator kernels live under the split allocator.paged module.
• Ignore stale or out-of-range allocator free ids before returning them to kvcached-managed allocators.

What this PR does not do

• It does not fully replace DeepSeek-V4 KV pools with kvcached-managed pools.
• It does not take over full-attention, SWA, compressed, or indexer pool lifecycle.
• It does not make performance claims; this is an accounting/budgeting correctness step.
• Full-attention pool takeover, compressed-attention pools, and broader DeepSeek-V4 pooling remain future work.

Related context

This PR is intentionally narrow and is no longer stacked on the other pending fixes. During multi-process / TP validation we also used or checked several related fixes, especially startup ordering and worker error propagation work from #336, #340, #341, #365, and #373. Those are separate PRs and are not included here.

Manual validation setup

Validated with DeepSeek-V4-Flash on two SGLang builds:

• SGLang PR #25960 (1c2857b0646f12a58a04072f0f3d6bd649f064f2, first released in v0.5.13)
• v0.5.13.post1 (85fd90072d1a9f2432842b03588f63b745e524e4), which exercises the newer split allocator.paged path

The runtime was configured to keep DeepSeek-V4 KV pools runtime-owned and disable the experimental kvcached bridge:

export ENABLE_KVCACHED=1
export KVCACHED_AUTOPATCH=1
export KVCACHED_REQUIRE=1
export KVCACHED_SGLANG_DSV4_RUNTIME_RESERVATION=1
export KVCACHED_SGLANG_DSV4_KV_POOL_BRIDGE=0

export SGLANG_OPT_DEEPGEMM_HC_PRENORM=0
export SGLANG_OPT_USE_TILELANG_MHC_PRE=0
export SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION=0
export SGLANG_DSA_FUSE_TOPK=false
export SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK=1

Server launch shape:

python3 -m sglang.launch_server \
  --model-path /model/DeepSeek-V4-Flash \
  --tokenizer-path /model/DeepSeek-V4-Flash \
  --served-model-name pjd-deepseek-v4-flash \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --kv-cache-dtype fp8_e4m3 \
  --mem-fraction-static 0.35 \
  --max-running-requests 16 \
  --chunked-prefill-size 8192 \
  --max-prefill-tokens 16384 \
  --prefill-max-requests 1 \
  --disable-radix-cache \
  --enable-cache-report \
  --fp8-gemm-backend cutlass \
  --fp4-gemm-backend cutlass \
  --moe-runner-backend marlin \
  --dsa-topk-backend torch \
  --cuda-graph-max-bs 16 \
  --disable-piecewise-cuda-graph

Router launch shape used for multi-worker validation:

python3 -m sglang_router.launch_router \
  --host 0.0.0.0 --port 19000 \
  --worker-urls http://127.0.0.1:18082 http://127.0.0.1:18083 \
  --policy cache_aware \
  --cache-threshold 0.8 \
  --balance-abs-threshold 8 \
  --balance-rel-threshold 1.2

Local validation

python -m pytest tests/test_sglang_elastic_allocator_patch.py tests/test_sglang_dsv4_runtime_reservation_patch.py -q
pre-commit run ruff --files kvcached/integration/sglang/autopatch.py kvcached/integration/sglang/patches.py kvcached/integration/sglang/interfaces.py tests/test_sglang_elastic_allocator_patch.py tests/test_sglang_dsv4_runtime_reservation_patch.py
pre-commit run isort --files kvcached/integration/sglang/autopatch.py kvcached/integration/sglang/patches.py kvcached/integration/sglang/interfaces.py tests/test_sglang_elastic_allocator_patch.py tests/test_sglang_dsv4_runtime_reservation_patch.py
pre-commit run codespell --files kvcached/integration/sglang/autopatch.py kvcached/integration/sglang/patches.py kvcached/integration/sglang/interfaces.py tests/test_sglang_elastic_allocator_patch.py tests/test_sglang_dsv4_runtime_reservation_patch.py

[Copilot]

Pull request overview

Adds the first SGLang DeepSeek-V4 integration step by accounting for runtime-owned DeepSeek-V4 KV/cache side pools as static reservations (without taking ownership), so kvcached’s device-memory budgeting doesn’t overbook colocated allocations. Also updates the SGLang elastic paged allocator patch to accept a runtime-provided num_new_pages, and adds targeted tests for both behaviors.

Changes:

• Introduce DeepSeek-V4 runtime-owned pool reservation collection/registration in the SGLang patch layer, and subtract those reservations from kvcached’s allocatable GPU budget.
• Extend the elastic paged allocator patch API to accept a precomputed num_new_pages (avoiding calling get_num_new_pages when provided).
• Add unit tests covering the allocator signature compatibility and DeepSeek-V4 reservation accounting behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/test_sglang_elastic_allocator_patch.py	Adds a regression test ensuring alloc_extend(..., num_new_pages=...) bypasses get_num_new_pages.
tests/test_sglang_dsv4_runtime_reservation_patch.py	Adds tests for DeepSeek-V4 per-pool reservation breakdown registration and budget subtraction in alloc_kv_cache.
kvcached/integration/sglang/patches.py	Implements DeepSeek-V4 runtime reservation patch and updates paged allocator patch to accept optional num_new_pages.
kvcached/integration/sglang/interfaces.py	Adds APIs for tracking runtime-owned reservations and subtracts them from the GPU budget during KV allocation.
kvcached/integration/sglang/autopatch.py	Registers the new DeepSeek-V4 runtime reservation patch in the SGLang autopatch sequence.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.