Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/getting-started/benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,3 +298,7 @@ Learn more about output options in the [Outputs documentation](../guides/outputs
## Authentication

When benchmarking against servers that require authentication (such as OpenAI's API), provide an API key in the backend configuration. See the [API Key Configuration](../guides/backends.md#api-key-configuration) section in the Backends documentation for details.

## Troubleshooting

See the [Troubleshooting guide](../guides/troubleshooting.md) for common issues.
2 changes: 1 addition & 1 deletion docs/getting-started/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,4 +90,4 @@ To use the vLLM Python backend (in-process inference), see [vLLM Python backend]

## Troubleshooting

If you encounter any issues during installation, ensure that your Python and pip versions meet the prerequisites. For further assistance, please refer to the [GitHub Issues](https://github.com/vllm-project/guidellm/issues) page or consult the [Documentation](https://github.com/vllm-project/guidellm/tree/main/docs).
If you encounter any issues during installation, ensure that your Python and pip versions meet the prerequisites. For common runtime errors (debug logging, tokenizer loading, macOS worker crashes), see the [Troubleshooting guide](../guides/troubleshooting.md). For further assistance, please refer to the [GitHub Issues](https://github.com/vllm-project/guidellm/issues) page.
2 changes: 2 additions & 0 deletions docs/guides/datasets.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ You can specify the tokenizer with `--tokenizer` and a configuration string. The

- `--tokenizer '{"kind":"huggingface_auto","model":"path/to/processor","load_kwargs":{"use_fast":false}}'`

If a HuggingFace model requires custom code to load, pass `trust_remote_code` in `load_kwargs`. See [Troubleshooting: trust_remote_code](troubleshooting.md#trust_remote_code-required-by-tokenizer).

If your dataset uses non-standard column names, you can use `--data-column-mapper` to map your columns to GuideLLM's expected column names. This is particularly useful when:

1. Your dataset uses different column names (e.g., `question` instead of `prompt`, `instruction` instead of `text_column`)
Expand Down
8 changes: 8 additions & 0 deletions docs/guides/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,4 +84,12 @@ Whether you're interested in understanding the system architecture, exploring su

[:octicons-arrow-right-24: Multimodal Guide](multimodal/index.md)

- :material-lifebuoy:{ .lg .middle } Troubleshooting

______________________________________________________________________

How to troubleshoot common errors.

[:octicons-arrow-right-24: Troubleshooting Guide](troubleshooting.md)

</div>
73 changes: 73 additions & 0 deletions docs/guides/troubleshooting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
weight: 15
---

# Troubleshooting

Find your symptom below, then follow the linked fix. For CLI syntax, see [Run a Benchmark](../getting-started/benchmark.md#cli-option-format).

| Symptom | Section |
| ----------------------------------------------------------- | ------------------------------------------------------------ |
| Requests fail or results look wrong | [Debug logging](#debug-logging) |
| Custom code error when loading a model's tokenizer. | [Tokenizer: trust_remote_code](#tokenizer-trust_remote_code) |
| `Worker process ... died unexpectedly (signal 11)` on macOS | [macOS worker crash](#macos-worker-crash-signal-11) |

## Debug logging

Enable debig output to inspect request handling and worker startup:

```bash
GUIDELLM__LOGGING__CONSOLE_LOG_LEVEL=DEBUG guidellm run ... --disable-progress
```

Run `guidellm env` to confirm the settings are being applied. The `--disable-progress` call is optional, but the interactive progress console can overwrite console log messages. Alternatively, you can use a file log as mentioned in the [logging guide](../developer/developing.md#logging) .

For all logging options (file output, log levels), see [Logging](../developer/developing.md#logging) in the development guide.

## Tokenizer: trust_remote_code

### Symptom

You get an error that looks like:

```text
The repository moonshotai/Kimi-K2.6 contains custom code which must be executed
to correctly load the model. You can inspect the repository content at
https://hf.co/moonshotai/Kimi-K2.6.
You can avoid this prompt in future by passing the argument `trust_remote_code=True`.
Comment thread
dbutenhof marked this conversation as resolved.
```

### Fix

If you fully trust the model, pass `trust_remote_code` through `--tokenizer` `load_kwargs`:

```bash
--tokenizer '{"kind":"huggingface_auto","load_kwargs":{"trust_remote_code":true}}'
Comment thread
dbutenhof marked this conversation as resolved.
```

Do not use this if you do not trust the model, as this allows code execution on your machine.

See [Datasets: Tokenizer](datasets.md#tokenizer) for other tokenizer options.

## macOS worker crash (signal 11)

### Symptom

```text
RuntimeError: Worker process group startup failed: Worker process <pid> died
unexpectedly (signal 11). Check system logs for details
```

You may also see repeated log lines such as:

```text
Worker process <pid> died unexpectedly (signal 11)
```

### Fix

GuideLLM defaults to `fork` multiprocessing, which can segfault on macOS. Use `spawn` instead:

```bash
GUIDELLM__MP_CONTEXT_TYPE=spawn guidellm run ...
```