Commit 8b4d41c
MUSA-0203: in-place query_start_loc update for captured loop replay
Add PR vllm-project#34880's matching in-place hunk for the captured replay path.
Without it the propose() loop reassigns:
common_attn_metadata.query_start_loc = self.arange[: batch_size + 1]
which creates a fresh tensor each call. The captured FA kernel baked
in the pre-capture pointer at boot time and reads stale data at replay
-> draft tokens past position 0 collapse (Pos 1-4 fell to 7/3/1/0.6%
on yeahdongcn70 cookbook).
PR vllm-project#34880 changes to:
common_attn_metadata.query_start_loc[: batch_size + 1] = self.arange[...]
which is an in-place write into the persistent buffer the captured
graph already references. Same fix for query_start_loc_cpu.
Test: reboot with VLLM_MUSA_DRAFT_FULL_WRAP=1 and rerun cookbook bench.
Expected: accept rate recovers, Pos 1-4 distribution matches eager.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>1 parent afd11cd commit 8b4d41c
1 file changed
Lines changed: 22 additions & 0 deletions
Lines changed: 22 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
192 | 192 | | |
193 | 193 | | |
194 | 194 | | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
195 | 216 | | |
196 | 217 | | |
197 | 218 | | |
| |||
360 | 381 | | |
361 | 382 | | |
362 | 383 | | |
| 384 | + | |
363 | 385 | | |
364 | 386 | | |
365 | 387 | | |
0 commit comments