[DNM][AMD] agentx-v0.4 rebased from commit chore/agentx-v0.4 commit 823189e

[seungrokj]

Summary

• Add qwen3.5-fp4-mi355x-sglang-agentic-hicache config: SGLang agentic-coding sweep with and without hicache offloading (TP2, EP1)
• Add minimaxm2.5-fp4-mi355x-vllm-agentic-lmcache config: vLLM agentic-coding sweep with lmcache
• Add new agentic benchmark scripts: minimaxm2.5_fp4_mi355x.sh, qwen3.5_fp4_mi355x.sh
• Update existing agentic scripts: glm5.1_fp4_mi355x.sh, kimik2.5_fp4_mi355x.sh, minimaxm2.5_fp8_mi355x.sh, qwen3.5_fp8_mi355x.sh
• Update launch_mi355x-amds.sh

Test plan

• Verify hicache/lmcache agentic configs run correctly on MI355X
• Confirm new agentic scripts launch without errors

🤖 Generated with Claude Code

---

Note

Medium Risk
Large CI config and container pin churn affects many GPU sweeps; PD disagg plus HiCache/Mooncake and nightly vLLM images add operational and regression risk on shared clusters.

Overview
Expands AMD MI355X benchmark coverage for agentic-coding and DeepSeek-V4 while rebasing image pins and keeping several fixed-seq-len entries aligned with origin/main via separate -agentic / -agentic-hicache YAML keys.

amd-master.yaml bumps SGLang/vLLM/Atom images, simplifies some Qwen3.5 fixed-seq search spaces, and adds net-new scenarios: DSv4 FP4 PD-disagg (dsv4-fp4-mi355x-sglang-disagg), agentic sweeps with CPU / HiCache / LMCache offload, and relocated agentic-only recipes (Kimi, MiniMax, GLM, Qwen) so main-line throughput configs stay unchanged. Some vLLM pins move to v0.21.0 or nightlies for ROCm KV offload; one top-level agentic block is commented out to limit sweep cost during image validation.

CI passes offloading from matrix config into e2e/sweep workflows and surfaces it in multinode job names.

Harness/runtime: default agentic trace loaders switch to 061526 corpora; new multi-node agentic launchers for DSR1/DSV4 disagg drive HiCache/Mooncake env. amd_utils gains HiCache config via bind-mounted env, trace_replay.sh for PD agentic runs, DeepSeek-V4-Pro in models.yaml, decoupled EP vs DP CLI flags, DSv4 bench --dsv4 framing, router circuit-breaker tweaks, and SGLang startup patches (disagg bootstrap desync, optional host-pool assert). Single-node agentic scripts add or extend DSv4/GLM HiCache and ATOM LMCache paths.

^{Reviewed by Cursor Bugbot for commit a493908. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[cursor]

vLLM uses wrong model

High Severity

The vLLM command serves "$MODEL" and omits --served-model-name, while the script downloads weights into MODEL_PATH and build_replay_cmd sends --model $MODEL to aiperf. That breaks the usual MODEL_PATH + served-name pairing used by sibling agentic scripts and can fail when MODEL is a Hub id but weights live under MODEL_PATH.

 

^{Reviewed by Cursor Bugbot for commit 01cc2af. Configure here.}

[cursor]

SGLang ignores MODEL_PATH

Medium Severity

SGLang is started with --model-path $MODEL and no --served-model-name, after the script may download into MODEL_PATH. Matrix jobs that set a local MODEL_PATH can still point the server at the Hub id, and the OpenAI model name may not match MODEL used by aiperf.

Additional Locations (1)

• benchmarks/single_node/agentic/qwen3.5_fp4_mi355x.sh#L123-L141

 

^{Reviewed by Cursor Bugbot for commit 01cc2af. Configure here.}

[cursor]

LMCache clone not idempotent

Medium Severity

The lmcache path runs git clone https://github.com/LMCache/LMCache.git unconditionally. With set -e, a second run in the same working directory exits when LMCache already exists, so lmcache agentic jobs fail on retry or reuse of the job cwd.

Additional Locations (1)

• benchmarks/single_node/agentic/minimaxm2.5_fp4_mi355x.sh#L149-L154

 

^{Reviewed by Cursor Bugbot for commit 01cc2af. Configure here.}

[cursor]

Server ignores MODEL_PATH

Medium Severity

Weights are downloaded into MODEL_PATH when the workflow sets that directory, but SGLang is started with --model-path $MODEL (Hub id) instead of MODEL_PATH. The server may load a different cache path than the one prepared for the job.

 

^{Reviewed by Cursor Bugbot for commit 32f5007. Configure here.}

[cursor]

DSv4 atom uncapped traces

Medium Severity

This new DSv4 ATOM agentic script sets WEKA_LOADER_OVERRIDE to the uncapped 060826 trace set, while peer MI355X agentic scripts in the same PR use 060226_256k to avoid ~1M-token traces that are rejected and skew sweeps.

 

^{Reviewed by Cursor Bugbot for commit 351e729. Configure here.}

[cursor]

MiniMax FP8 launcher regressed

High Severity

The MI355X MiniMax FP8 agentic launcher was replaced with a Kimi-style vLLM recipe. Existing minimaxm2.5-fp8-mi355x-vllm-agentic jobs (TP4/EP4, offloading=cpu) lose the prior --max-model-len, ROCM_AITER_UNIFIED_ATTN backend, MODEL_PATH-based serve, and SimpleCPU offload wiring they depended on.

 

^{Reviewed by Cursor Bugbot for commit faba18f. Configure here.}

[cursor]

DP max-running requests wrong

Medium Severity

When DP_ATTENTION=true, the script computes PER_ENGINE_MAX_RUNNING as CONC/TP for per-engine limits, but the server is started with --max-running-requests ${CONC}. Each DP engine may accept too many sequences versus the harness load-balancing assumption.

 

^{Reviewed by Cursor Bugbot for commit 76d90e0. Configure here.}

[cursor]

Wrong model path for serve

Medium Severity

The script downloads weights into MODEL_PATH when set, but sglang serve uses --model-path $MODEL (Hub id) instead of "$MODEL_PATH". Runs that pre-stage a local directory can ignore the prepared path and rely on a different cache location.

 

^{Reviewed by Cursor Bugbot for commit 4ebc4e2. Configure here.}

[cursor]

Disagg agentic uses wrong runner

High Severity

dsr1-fp4-mi355x-sglang-disagg-agentic-hicache sets runner: mi355x while sibling PD-disagg entries (including dsv4-fp4-mi355x-sglang-disagg-agentic-hicache) use runner: mi355x-disagg. Multinode jobs use runs-on: ${{ inputs.runner }}, so the DSR1 agentic disagg matrix likely schedules on the wrong runner class.

Additional Locations (1)

• .github/configs/amd-master.yaml#L2848-L2857

 

^{Reviewed by Cursor Bugbot for commit a9e1304. Configure here.}

[cursor]

DSv4 HiCache ratio inconsistent

Low Severity

Both DSv4 MI355X agentic SGLang launchers share the same HiCache comment for TP≥8, but dsv4_fp4_mi355x_sglang.sh defaults DEFAULT_HICACHE_RATIO to 2 while dsv4_fp4_mi355x.sh uses 8, so identical YAML sweeps get different CPU tier sizing.

Additional Locations (1)

• benchmarks/single_node/agentic/dsv4_fp4_mi355x.sh#L73-L75

 

^{Reviewed by Cursor Bugbot for commit a9e1304. Configure here.}

[cursor]

CPU offload divide by zero

Medium Severity

TOTAL_CPU_DRAM_PARTITION_GB uses $((TOTAL_CPU_DRAM_GB / (8 / TP))). For TP greater than 8, bash evaluates 8 / TP as 0 and arithmetic expansion errors on divide-by-zero.

Additional Locations (1)

• benchmarks/single_node/agentic/minimaxm2.5_fp8_mi355x.sh#L124-L126

 

^{Reviewed by Cursor Bugbot for commit a9e1304. Configure here.}

[cursor]

Disagg MoE token overrides removed

Medium Severity

This change comments out assignments that set MORI_MOE_MAX_INPUT_TOKENS_PREFILL and MORI_MOE_MAX_INPUT_TOKENS_DECODE for DP+EP and MTP decode paths, while launch commands still conditionally export those variables. Disagg sweeps that relied on the computed caps may run with unset MoE input limits.

 

^{Reviewed by Cursor Bugbot for commit c21ad06. Configure here.}

[cursor]

Server ignores resolved MODEL_PATH

Medium Severity

job.slurm now resolves and exports a canonical MODEL_PATH (caller path, hf_dir, or MODEL_DIR/MODEL_NAME), but server_sglang.sh still launches with --model-path $MODEL_DIR/$MODEL_NAME. When the resolved path differs from that join, prefill/decode can fail to load weights or load from the wrong directory.

Additional Locations (1)

• benchmarks/multi_node/amd_utils/job.slurm#L209-L242

 

^{Reviewed by Cursor Bugbot for commit c7f269e. Configure here.}

[cursor]

Custom all-reduce flag unused

Medium Severity

DISABLE_CUSTOM_ALL_REDUCE is threaded into the container from job.slurm, and the DSR1 disagg agentic recipe defaults it to 1 for an Aiter fault workaround, but prefill/decode launch commands never append --disable-custom-all-reduce.

 

^{Reviewed by Cursor Bugbot for commit b5626fb. Configure here.}

[cursor]

Ignores TOTAL_CPU_DRAM_GB env

Medium Severity

The scripts require TOTAL_CPU_DRAM_GB via check_env_vars, then hardcode TOTAL_CPU_DRAM_GB=3000 inside the cpu and lmcache branches. Workflow-supplied offload memory sizing is discarded and partition math uses the fixed constant instead.

Additional Locations (1)

• benchmarks/single_node/agentic/dsv4_fp4_mi355x_atom.sh#L113-L155

 

^{Reviewed by Cursor Bugbot for commit 73756ab. Configure here.}

[cursor]

Disagg trace corpus not 256k capped

Medium Severity

Multinode agentic replay hardcodes WEKA_LOADER_OVERRIDE=semianalysis_cc_traces_weka_061526 (full corpus). Single-node agentic scripts use _256k variants for non-DSv4 models, while benchmark_lib.sh now defaults non-dsv4 recipes to 061526_256k. Disagg agentic jobs can load traces that exceed max_model_len and fail or skew results.

 

^{Reviewed by Cursor Bugbot for commit 219bcc9. Configure here.}

[cursor]

Replay failures do not fail job

Medium Severity

When any concurrency sweep in trace_replay.sh fails, the script only prints a warning and exits 0, so SLURM and CI can treat a broken agentic disagg run as success without a valid aggregate result.

 

^{Reviewed by Cursor Bugbot for commit 7c2bcbc. Configure here.}

[cursor]

DSv4 atom script is Kimi

High Severity

The added dsv4_fp4_mi355x_atom.sh is a Kimi-K2.5 ATOM/vLLM offload recipe (comments, LMCache clone, --kv_offloading_backend, atom.entrypoints.openai_server) and does not implement DeepSeek-V4-Pro ATOM serving despite the filename and PR intent.

 

^{Reviewed by Cursor Bugbot for commit 7c2bcbc. Configure here.}