Skip to content

Commit 630e804

Browse files
committed
review: address second-pass docs and code comments
- Replace bare assert in both synthetic deserializers with an explicit isinstance check + RuntimeError, matching guidellm's style for unexpected-type guards. - docs/guides/multimodal/index.md: expand "VLM" to "Vision-Language Model (VLM)" on the Synthetic Vision card to avoid the VLM/vLLM/LLM visual collision. - docs/guides/multimodal/synthetic_vision.md: - drop the "wire-size pin" phrasing from the bitrate example - "pin"/"pinned" -> "specify"/"fixed" in the video_bitrate bullet - rewrite the ffmpeg/PIL note to just warn about byte-level variability across versions, instead of recommending users modify the uv.lock file - pyproject.toml unchanged; uv.lock regenerated via `uv sync --extra vision` so it tracks the vision-extra dependency closure. Signed-off-by: Zakaria el hjouji <zakaria@overshoot.ai>
1 parent ca174c3 commit 630e804

5 files changed

Lines changed: 82 additions & 8 deletions

File tree

docs/guides/multimodal/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ Ensure you have a running inference server and model compatible with the OpenAI
5353

5454
______________________________________________________________________
5555

56-
Generate images and short videos on the fly to benchmark VLM serving configurations without bringing your own dataset. Covers the `synthetic_image` and `synthetic_video` `--data` types.
56+
Generate images and short videos on the fly to benchmark Vision-Language Model (VLM) serving configurations without bringing your own dataset. Covers the `synthetic_image` and `synthetic_video` `--data` types.
5757

5858
[:octicons-arrow-right-24: Synthetic Vision Guide](synthetic_vision.md)
5959

docs/guides/multimodal/synthetic_vision.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ guidellm benchmark run \
6666
--data "type=synthetic_video,width=854,height=480,frames=6,fps=1,text_tokens=64,output_tokens=128"
6767
```
6868

69-
A twelve-frame 720p clip at 3 fps with a wire-size pin:
69+
A twelve-frame 720p clip at 3 fps with an explicit h264 target bitrate:
7070

7171
```bash
7272
guidellm benchmark run \
@@ -80,7 +80,7 @@ guidellm benchmark run \
8080
- `height`: Height of the generated video in pixels. The same `resolution` / `aspect_ratio` shorthands as for synthetic image apply.
8181
- `frames`: Number of frames in the clip.
8282
- `fps`: Frames per second. Combined with `frames`, this also determines the clip duration.
83-
- `video_bitrate`: Optional h264 target bitrate (e.g. `1M`, `500k`) — useful when wire size needs to be pinned across runs.
83+
- `video_bitrate`: Optional h264 target bitrate (e.g. `1M`, `500k`) — useful when you want to specify a fixed wire size across runs.
8484
- `content`: Per-row clip content. `gradient` (default) emits a seeded gradient with a coordinate warp so each clip compresses similarly to real video; `noise` emits uniform random pixels for worst-case wire size.
8585
- `text_tokens`: Average number of tokens in the accompanying text prompt; same `stdev` / `min` / `max` suffixes as synthetic image. `prompt_tokens` is accepted as an alias.
8686
- `output_tokens`: Average number of tokens the model should generate; same `stdev` / `min` / `max` suffixes apply.
@@ -90,4 +90,4 @@ guidellm benchmark run \
9090

9191
- A processor/tokenizer is required for the text portion of the request. By default the model passed in or retrieved from the server is used; otherwise specify one with `--processor`.
9292
- Per-row seeded gradients produce byte-different payloads on every request, which bypasses vLLM's multimodal preprocessor cache. If you want to deliberately hit the cache, set `content=solid` or pin a fixed `seed` and `samples`.
93-
- Reproducibility of exact mp4 bytes depends on the installed `ffmpeg` and `PIL` versions. Pin via the lockfile if you compare runs across machines.
93+
- The exact mp4 bytes produced for a given seed depend on the installed `ffmpeg` and `PIL` versions. Output token counts and request shape stay stable across versions, but if you are comparing byte-level outputs or wire-size measurements across machines, expect small variation.

src/guidellm/data/deserializers/synthetic_image.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,8 +75,12 @@ def _build_prompt(
7575
text = unique + faker.text(max_nb_chars=num_chars)
7676
token_ids = processor.encode(text)
7777
decoded = processor.decode(token_ids[:token_count], skip_special_tokens=True)
78-
assert isinstance(decoded, str) # noqa: S101
79-
return decoded
78+
if isinstance(decoded, str):
79+
return decoded
80+
raise RuntimeError(
81+
"Processor.decode returned a non-string value while generating "
82+
"synthetic image prompt text."
83+
)
8084

8185
def __iter__(self) -> Iterator[tuple[int, dict[str, Any]]]:
8286
iter_seed = self.random_seed + self.iteration_count

src/guidellm/data/deserializers/synthetic_video.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -69,8 +69,12 @@ def _build_prompt(
6969
text = unique + faker.text(max_nb_chars=num_chars)
7070
token_ids = processor.encode(text)
7171
decoded = processor.decode(token_ids[:token_count], skip_special_tokens=True)
72-
assert isinstance(decoded, str) # noqa: S101
73-
return decoded
72+
if isinstance(decoded, str):
73+
return decoded
74+
raise RuntimeError(
75+
"Processor.decode returned a non-string value while generating "
76+
"synthetic video prompt text."
77+
)
7478

7579
def __iter__(self) -> Iterator[tuple[int, dict[str, Any]]]:
7680
iter_seed = self.random_seed + self.iteration_count

uv.lock

Lines changed: 66 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)