Add troubleshooting guide#860
Conversation
Generated-by: Cursor AI Claude Opus 4.6 Signed-off-by: Jared O'Connell <joconnel@redhat.com>
dbutenhof
left a comment
dbutenhof
left a comment
There was a problem hiding this comment.
I might be tempted to add another troubleshooting scenario we're on the cusp of solving --
Problem: --profile kind=sweep --constraint kind=max_requests,count=1000 runs a long time. (E.g., "A benchmark configured with --max-requests=1000 --rate-type=sweep took ~7 hours to complete on H100 GPUs, when users expected it to complete in minutes.")
Solution: The constraint is applied separately to each benchmark strategy, and sweep's first "synchronous" strategy will run 1000 requests sequentially. Specify a smaller request limit, or limit each benchmark strategy by duration rather than by count, with --constraint kind=max_duration,seconds=60.
| | Symptom | Section | | ||
| | ----------------------------------------------------------- | ------------------------------------------------------------ | | ||
| | Requests fail or results look wrong | [Debug logging](#debug-logging) | | ||
| | `trust_remote_code=True` when loading a tokenizer | [Tokenizer: trust_remote_code](#tokenizer-trust_remote_code) | |
There was a problem hiding this comment.
That doesn't sound like a "symptom" -- it's the solution. The "symptom" is more like the statement at your link: "a HuggingFace model requires custom code to load".
| Enable verbose output to inspect request handling and worker startup: | ||
|
|
||
| ```bash | ||
| GUIDELLM__LOGGING__CONSOLE_LOG_LEVEL=DEBUG guidellm run ... |
There was a problem hiding this comment.
You might want to suggest disabling console output, which can make it hard to read debug logs -- or use GUIDELLM__LOGGING__LOG_FILE to log to a file??
| The repository moonshotai/Kimi-K2.6 contains custom code which must be executed | ||
| to correctly load the model. You can inspect the repository content at | ||
| https://hf.co/moonshotai/Kimi-K2.6. | ||
| You can avoid this prompt in future by passing the argument `trust_remote_code=True`. |
There was a problem hiding this comment.
This is the fix, not part of the "symptom".
| Pass `trust_remote_code` through `--tokenizer` `load_kwargs`: | ||
|
|
||
| ```bash | ||
| --tokenizer '{"kind":"huggingface_auto","load_kwargs":{"trust_remote_code":true}}' |
There was a problem hiding this comment.
Although I'd hesitate to put this in the documentation, I'd be really tempted to shortcut that with kind=huggingface_auto,load_kwargs.trust_remote_code=true 😁
2 participants
Summary
Adds a troubleshooting section to the docs
Details
Talks about
Test Plan
Use of AI
git log
commit 57b86e6
Author: Jared O'Connell joconnel@redhat.com
Date: Thu Jun 25 14:24:05 2026 -0400
Generated-by: Cursor AI Claude Opus 4.6
Signed-off-by: Jared O'Connell joconnel@redhat.com