Skip to content

Commit 80952de

Browse files
committed
Update GB300 FP4 GLM5 low-latency sweep
1 parent 3523798 commit 80952de

1 file changed

Lines changed: 5 additions & 5 deletions

File tree

recipes/gb300-fp4/glm5.yaml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -186,8 +186,8 @@ zip_override_8k1k_lowlat:
186186
resources:
187187
prefill_nodes: 1
188188
prefill_workers: 1
189-
decode_nodes: [3, 5, 9, 15]
190-
decode_workers: [3, 5, 9, 15]
189+
decode_nodes: [3, 5, 9, 15, 17]
190+
decode_workers: [3, 5, 9, 15, 17]
191191
backend:
192192
sglang_config:
193193
decode:
@@ -199,13 +199,13 @@ zip_override_8k1k_lowlat:
199199

200200
moe-runner-backend: "flashinfer_trtllm"
201201

202-
max-running-requests: 4096
203-
cuda-graph-max-bs: 4096
202+
max-running-requests: [128, 64, 32, 16, 1]
203+
cuda-graph-max-bs: [128, 64, 32, 16, 1]
204204

205205
benchmark:
206206
isl: 8192
207207
osl: 1024
208-
concurrencies: ["1024"]
208+
concurrencies: ["128", "64", "32", "16", "8"]
209209

210210

211211

0 commit comments

Comments
 (0)