Validated vllm 0.26 / torch 2.11 waypoint (runs on CUDA 12.8 drivers)

[abhishekraok]

Claude Code · session 09974a79 (claude.ai/code), for @abhishekraok

What this is

A fully GPU-validated vllm 0.19.1 → 0.26.0 / torch 2.10 → 2.11 upgrade that runs on the current CUDA 12.8-driver fleet (jupiter/ceres/saturn/titan, not just holmes). Kept as a draft on purpose — see "Relationship to #1873" below for why this may never merge, and under what conditions it should.

The key enabling fact: PyPI vllm wheels are CUDA 13 builds from 0.26 onward, but vllm publishes cu129 wheels at wheels.vllm.ai, and CUDA 12.x minor-version compatibility runs them fine on 12.8 drivers — verified on real H100 nodes with real training, not just imports.

Relationship to torch-213-upgrade (#1873)

@gregorybchris's #1873 targets the right endpoint (torch 2.13 / vllm 0.28 / single cu130 build) and supersedes the vllm-integration parts of this branch — vllm 0.28 deleted the 0.26 weight-sync API this branch migrates to. This PR exists for two reasons:

1. Insurance: Upgrade torch to 2.13 #1873 is hard-gated on the fleet-wide CUDA 13 driver rollout (currently end of September, a date that has already slipped once for robotics workloads). If the rollout slips again, this branch is the only newer-vllm path that runs on the existing fleet, and it is merge-ready.
2. Quarry: several fixes here apply to Upgrade torch to 2.13 #1873 verbatim and were only discoverable by running on GPU (details posted as a comment there): the torch ≥2.11 ConfigModule/Ray-cloudpickle pickling patch, the tokenization-cache cold-start timeout, and the two validated GRPO smoke configs as a validation recipe.

Merge this only if someone needs vllm ≥0.26 on the 12.8-driver clusters before #1873 lands. Otherwise close it once #1873 is validated. The datasets pin here duplicates #1867 and drops out on rebase once that merges.

Changes

• vllm 0.19.1 → 0.26.0 (+cu129 on the cuda12 stack, +cu130 on cuda13), which moves torch to 2.11.0 — the version the OLMoE3 KDA MoE requires (CUBLAS_STATUS_NOT_INITIALIZED on 2.10), so this also dissolves the spike/s004-moe override-dependencies hack.
• transformers → 5.16.1 (vllm floor; also fixes the override-dependencies floor to >=5.5.3 so the lock stops silently overriding vllm's own requirement), openai → 3.x, datasets >=4.8.4,<5 (torchvision 0.26 removed VideoReader; same as Bump datasets to >=4.8.4,<5 #1867), torchcodec added per-stack (+cpu on cuda12 — no cu128 builds exist past 0.14 — and +cu130 on cuda13).
• flash-attn-4 → beta19, the only release band compatible with vllm 0.26's exact nvidia-cutlass-dsl==4.6.0 + apache-tvm-ffi==0.1.10 pins (older betas call the removed cute.core.ThrMma; beta20+ pin 4.6.0.dev0 or tvm-ffi>=0.1.12). flash-attn 2 / FA3 stay on torch2.10-built wheels — no torch 2.11 builds exist anywhere (wheel builders went 2.10 → 2.13); cross-ABI loading is validated by both GPU smokes below.
• vllm 0.26 API migrations in vllm_utils.py: IPCTrainerSendWeightsArgs.mode → send_mode; update_weights now requires a start_weight_update/finish_weight_update bracket. Qwen3XMLToolParser → Qwen3EngineToolParser in environments/tools/parsers.py.
• torch 2.11 ConfigModule pickling patch: torch.utils._config_module.ConfigModule instances (e.g. torch.distributed.config) subclass ModuleType but fail Ray's cloudpickle exact-type check, killing every Ray actor export. Fixed with a __reduce__ that re-imports by module name.
• olmo_adapter: config.num_labels is int | None in transformers 5.16 — narrowed with an explicit error.
• uv plumbing: required-environments so xgrammar resolves to versions with x86_64 wheels; new vllm-cu129/vllm-cu130 indexes.
• SFT running guide: documents the per-model-family torch/CUDA constraints (dense/hybrid vs KDA MoE).

Validation

Runs:

1. Full CPU suite: 738 passed, 0 failed.
2. Single-GPU GRPO smoke, IPC weight path, real training (gsm8k 0.69): Beaker
3. 2-node GRPO smoke, NCCL weight path, 17 weight syncs across 2 nodes, code_correct_rate 0.86: Beaker

scripts/test/run_gpu_pytest.sh has not been run yet; it will be before this leaves draft, if it ever does.

🤖 Generated with Claude Code

[github-actions]

Documentation Changes Detected

📄 algorithms/sft_running_guide/index.html

--- site-base/algorithms/sft_running_guide/index.html	2026-09-02 18:38:55.813025507 +0000
+++ site-pr/algorithms/sft_running_guide/index.html	2026-09-02 18:38:34.049117960 +0000
@@ -1448,6 +1448,15 @@
   <code>TransformerConfig</code> preset; there is no fallback if it does not. Architectures
   olmo-core has no preset or HF weight conversion for need both written first —
   see the Olmo-Hybrid report above for what that costs.</li>
+<li><strong>Torch/CUDA constraints per model family.</strong> Dense Olmo 3 and Olmo-Hybrid train
+  on the default torch 2.10 / cu128 image, which runs on every WEKA cluster. The
+  OLMoE3 KDA MoE requires torch 2.11: on 2.10 the KDA forward dies with
+  <code>CUBLAS_STATUS_NOT_INITIALIZED</code> (an 18M-param layer at 0.36 GiB reproduces it —

Showing first 10 lines of diff for each changed file (up to 5 files, excluding search indices).

[github-actions]

Documentation Changes Detected

📄 algorithms/sft_running_guide/index.html

--- site-base/algorithms/sft_running_guide/index.html	2026-09-02 18:40:33.416938100 +0000
+++ site-pr/algorithms/sft_running_guide/index.html	2026-09-02 18:39:50.397995091 +0000
@@ -1448,6 +1448,15 @@
   <code>TransformerConfig</code> preset; there is no fallback if it does not. Architectures
   olmo-core has no preset or HF weight conversion for need both written first —
   see the Olmo-Hybrid report above for what that costs.</li>
+<li><strong>Torch/CUDA constraints per model family.</strong> Dense Olmo 3 and Olmo-Hybrid train
+  on the default torch 2.10 / cu128 image, which runs on every WEKA cluster. The
+  OLMoE3 KDA MoE requires torch 2.11: on 2.10 the KDA forward dies with
+  <code>CUBLAS_STATUS_NOT_INITIALIZED</code> (an 18M-param layer at 0.36 GiB reproduces it —

Showing first 10 lines of diff for each changed file (up to 5 files, excluding search indices).