10.8.1 breaks MTP with external MTP heads on user models

[alucryd]

Platform

Linux/Arch

Lemonade Version

10.8.1

GPU / APU Model

AMD Ryzen AI Max 395+

Component

llama.cpp

Bug Description

Lemonade now forbids passing -md to llama.cpp, without providing a way to pass the heads in user_models.json, essentially making us unable to use a custom Gemma 4 or other models with separate heads.

Steps to Reproduce

1. Download the QAT versions of Gemma 4 from Unsloth
2. Download the associated MTP checkpoints
3. Try to load the model and pass the MTP head as argument

Expected vs Actual Behavior

Lemonade should allow passing -md with user models, or at least use the same format in both server_models.json and user_models.json so we have at least one way to pass an MTP head.

Log Output

Additional Context

No response

[bitgamma]

this is an intentional change and works the same as for the model file itself and mmproj file. You can specify the draft model by adding it to the model definition itself, as "draft" checkpoint. Example of json that can be imported (either via the webui or with lemonade import and defines the draft checkpoint:

{
  "checkpoints": {
    "draft": "unsloth/gemma-4-26B-A4B-it-qat-GGUF:mtp-gemma-4-26B-A4B-it.gguf",
    "main": "unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL",
    "mmproj": "unsloth/gemma-4-26B-A4B-it-qat-GGUF:mmproj-BF16.gguf"
  },
  "labels": [
    "custom",
    "vision",
    "mtp",
    "tool-calling"
  ],
  "recipe": "llamacpp",
  "recipe_options": {
    "ctx_size": 262144,
    "llamacpp_args": "--temp 1.0 --top_p 0.95 --top_k 64 --spec-type draft-mtp --spec-draft-n-max 4 --presence_penalty 0.0 --reasoning on",
    "llamacpp_backend": "vulkan",
    "merge_args": true,
    "pinned": false
  },
  "size": 14.4,
  "model_name": "user.gemma-4-26B-A4B-it-qat-GGUF-UD-Q4_K_XL"
}

[alucryd]

I did try this, and my user_models.json file became unreadable, none of my user models came up in the web ui. I'm quite sure syntax was correct, but I'll give it another try.