Skip to content

Commit 12c0e9b

Browse files
committed
Add details on endpoints config
1 parent 001917f commit 12c0e9b

1 file changed

Lines changed: 45 additions & 3 deletions

File tree

multimodal/qwen3-vl/README.md

Lines changed: 45 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -46,26 +46,68 @@ pip install .
4646

4747
### Configure the benchmark
4848

49-
Example configs live under [endpoints/examples/08_Qwen3-VL-235B-A22B_Example](https://github.com/mlcommons/endpoints/tree/381d13bbd27d6d52306813a51dc4e44295222d7e/examples/08_Qwen3-VL-235B-A22B_Example). Set the endpoint URL in the YAML file to match your server address and port:
49+
Example configs live under [endpoints/examples/08_Qwen3-VL-235B-A22B_Example](https://github.com/mlcommons/endpoints/tree/381d13bbd27d6d52306813a51dc4e44295222d7e/examples/08_Qwen3-VL-235B-A22B_Example).
50+
51+
#### Fields that the submitter **should** update to match their server status:
52+
53+
- Served model name:
54+
55+
```yaml
56+
model_params:
57+
name: "Qwen/Qwen3-VL-235B-A22B-Instruct"
58+
```
59+
60+
- Endpoints url and port:
5061
5162
```yaml
5263
endpoint_config:
5364
endpoints:
5465
- "http://localhost:8000"
5566
```
5667
68+
#### Fields that the submitter **may** customize for performance tuning:
69+
70+
- Target_qps (for server and interactive mode):
71+
72+
```yaml
73+
load_pattern:
74+
type: "poisson"
75+
target_qps: 6.5
76+
```
77+
78+
- Client worker related settings:
79+
80+
```yaml
81+
client:
82+
num_workers: 5
83+
transport:
84+
type: zmq
85+
recv_buffer_size: 16777216
86+
send_buffer_size: 16777216
87+
max_connections: 1000
88+
worker_initialization_timeout: 120
89+
```
90+
91+
#### Fileds that the submitter **MUST NOT** change for valid results:
92+
93+
- Sampling parameters that specified in scection [Reference Implementation Specification](#reference-implementation-specification)
94+
95+
- Datasets (Neither performance or accuracy dataset)
96+
5797
### Run the benchmark
5898
59-
Launch offline or server scenarios:
99+
Launch the offline scenario:
60100
61101
```bash
62102
uv run inference-endpoint benchmark from-config \
63103
-c examples/08_Qwen3-VL-235B-A22B_Example/offline_qwen3_vl_235b_a22b_shopify.yaml
64104
```
65105

106+
Launch the server scenario:
107+
66108
```bash
67109
uv run inference-endpoint benchmark from-config \
68-
-c examples/08_Qwen3-VL-235B-A22B_Example/online_qwen3_vl_235b_a22b_shopify.yaml
110+
-c examples/08_Qwen3-VL-235B-A22B_Example/server_qwen3_vl_235b_a22b_shopify.yaml
69111
```
70112

71113
Launch the interactive scenario:

0 commit comments

Comments
 (0)