|
| 1 | +--- |
| 2 | +weight: 15 |
| 3 | +--- |
| 4 | + |
| 5 | +# Troubleshooting |
| 6 | + |
| 7 | +Find your symptom below, then follow the linked fix. For CLI syntax, see [Run a Benchmark](../getting-started/benchmark.md#cli-option-format). |
| 8 | + |
| 9 | +| Symptom | Section | |
| 10 | +| ----------------------------------------------------------- | ------------------------------------------------------------ | |
| 11 | +| Requests fail or results look wrong | [Debug logging](#debug-logging) | |
| 12 | +| Custom code error when loading a model's tokenizer. | [Tokenizer: trust_remote_code](#tokenizer-trust_remote_code) | |
| 13 | +| `Worker process ... died unexpectedly (signal 11)` on macOS | [macOS worker crash](#macos-worker-crash-signal-11) | |
| 14 | + |
| 15 | +## Debug logging |
| 16 | + |
| 17 | +Enable debig output to inspect request handling and worker startup: |
| 18 | + |
| 19 | +```bash |
| 20 | +GUIDELLM__LOGGING__CONSOLE_LOG_LEVEL=DEBUG guidellm run ... --disable-progress |
| 21 | +``` |
| 22 | + |
| 23 | +Run `guidellm env` to confirm the settings are being applied. The `--disable-progress` call is optional, but the interactive progress console can overwrite console log messages. Alternatively, you can use a file log as mentioned in the [logging guide](../developer/developing.md#logging) . |
| 24 | + |
| 25 | +For all logging options (file output, log levels), see [Logging](../developer/developing.md#logging) in the development guide. |
| 26 | + |
| 27 | +## Tokenizer: trust_remote_code |
| 28 | + |
| 29 | +### Symptom |
| 30 | + |
| 31 | +You get an error that looks like: |
| 32 | + |
| 33 | +```text |
| 34 | +The repository moonshotai/Kimi-K2.6 contains custom code which must be executed |
| 35 | +to correctly load the model. You can inspect the repository content at |
| 36 | +https://hf.co/moonshotai/Kimi-K2.6. |
| 37 | +You can avoid this prompt in future by passing the argument `trust_remote_code=True`. |
| 38 | +``` |
| 39 | + |
| 40 | +### Fix |
| 41 | + |
| 42 | +If you fully trust the model, pass `trust_remote_code` through `--tokenizer` `load_kwargs`: |
| 43 | + |
| 44 | +```bash |
| 45 | +--tokenizer '{"kind":"huggingface_auto","load_kwargs":{"trust_remote_code":true}}' |
| 46 | +``` |
| 47 | + |
| 48 | +Do not use this if you do not trust the model, as this allows code execution on your machine. |
| 49 | + |
| 50 | +See [Datasets: Tokenizer](datasets.md#tokenizer) for other tokenizer options. |
| 51 | + |
| 52 | +## macOS worker crash (signal 11) |
| 53 | + |
| 54 | +### Symptom |
| 55 | + |
| 56 | +```text |
| 57 | +RuntimeError: Worker process group startup failed: Worker process <pid> died |
| 58 | +unexpectedly (signal 11). Check system logs for details |
| 59 | +``` |
| 60 | + |
| 61 | +You may also see repeated log lines such as: |
| 62 | + |
| 63 | +```text |
| 64 | +Worker process <pid> died unexpectedly (signal 11) |
| 65 | +``` |
| 66 | + |
| 67 | +### Fix |
| 68 | + |
| 69 | +GuideLLM defaults to `fork` multiprocessing, which can segfault on macOS. Use `spawn` instead: |
| 70 | + |
| 71 | +```bash |
| 72 | +GUIDELLM__MP_CONTEXT_TYPE=spawn guidellm run ... |
| 73 | +``` |
0 commit comments