Reporting this here because the reproduction lives in sd.cpp, and because one part of it is sd.cpp's to fix.
Setup
- Radeon RX 7900 XTX, RADV (Mesa 25.2.8), kernel 7.0.0-31, 30 GiB system RAM
mem_info_gtt_total = 15591 MiB (amdgpu defaults GTT to about half of system RAM)- sd.cpp at
c678dfe, built with -DSD_VULKAN=ON
- Qwen-Image-2.1 Q8_0 + Qwen3-VL-8B-Instruct Q8_0 +
qwen_image_2.1_vae_bf16
sd-cli --diffusion-model qwen_image_2.1-Q8_0.gguf \
--vae qwen_image_2.1_vae_bf16.safetensors \
--llm Qwen3VL-8B-Instruct-Q8_0.gguf \
--backend vulkan0 --fa --offload-to-cpu \
--steps 20 --cfg-scale 6.0 --sampling-method euler -H 1024 -W 1024 -p "..."
radv/amdgpu: Not enough memory for command submission.
[ERROR] ggml_vulkan: device lost on Vulkan0
[ERROR] wan_vae graph execution failed on Vulkan0: vk::Queue::submit: ErrorDeviceLost
[WARN ] backend_fit.cpp:501 - VAE decode failed (likely out of memory); retrying with spatial tiling
This is not VRAM exhaustion — the card has 24 GiB and peak usage was ~8 GiB. --offload-to-cpu pins about 16.7 GiB of weights as host-visible Vulkan buffers, and amdgpu's GTT budget is 15.23 GiB. I filed the ggml side as ggml-org/llama.cpp#29213 with the measurements; it should reach this repo when the ggml submodule is next synced.
The part that is sd.cpp's
backend_fit.cpp:501 retries the VAE decode with spatial tiling after the device has been lost, so the retry cannot succeed — the whole context is gone. It would be worth checking whether the device is still alive before retrying, and failing with the original error if it is not; right now the log reads as if tiling was tried and did not help, when in fact tiling was never really attempted.
Starting with --vae-tiling from the outset does work on the same machine: the decode compute buffer drops from 11322 MB to 2556 MB and the decode takes 4.6 s.
Two smaller things
--vae-tile-size is in latent units, not pixels, and the help text does not say so. At 1024×1024 the latent is 128×128, so --vae-tile-size 256x256 yields num tiles : 1, 1 — tiling silently does nothing. I lost a while to this while assuming tiling could not help.
--fa vs --diffusion-fa for Qwen-Image-2.1. The LLM text encoder path uses ctx->flash_attn_enabled, which --diffusion-fa does not set, so a command line carried over from a FLUX setup leaves flash attention off for the text encoder without any indication. Possibly worth a note in docs/qwen_image_2.1.md.
For reference, what works on this machine
--backend vulkan0 --fa --vae-tiling with no --offload-to-cpu: the weights (16.7 GiB) go straight to VRAM, GTT is not involved at all, and 1024×1024 takes 139 s (text encode 18 s, sampling 116 s, VAE 4.5 s), peak 17.74 GiB. With a reference image, 272 s and peak 18.93 GiB.
Reporting this here because the reproduction lives in sd.cpp, and because one part of it is sd.cpp's to fix.
Setup
mem_info_gtt_total= 15591 MiB (amdgpu defaults GTT to about half of system RAM)c678dfe, built with-DSD_VULKAN=ONqwen_image_2.1_vae_bf16This is not VRAM exhaustion — the card has 24 GiB and peak usage was ~8 GiB.
--offload-to-cpupins about 16.7 GiB of weights as host-visible Vulkan buffers, and amdgpu's GTT budget is 15.23 GiB. I filed the ggml side as ggml-org/llama.cpp#29213 with the measurements; it should reach this repo when the ggml submodule is next synced.The part that is sd.cpp's
backend_fit.cpp:501retries the VAE decode with spatial tiling after the device has been lost, so the retry cannot succeed — the whole context is gone. It would be worth checking whether the device is still alive before retrying, and failing with the original error if it is not; right now the log reads as if tiling was tried and did not help, when in fact tiling was never really attempted.Starting with
--vae-tilingfrom the outset does work on the same machine: the decode compute buffer drops from 11322 MB to 2556 MB and the decode takes 4.6 s.Two smaller things
--vae-tile-sizeis in latent units, not pixels, and the help text does not say so. At 1024×1024 the latent is 128×128, so--vae-tile-size 256x256yieldsnum tiles : 1, 1— tiling silently does nothing. I lost a while to this while assuming tiling could not help.--favs--diffusion-fafor Qwen-Image-2.1. The LLM text encoder path usesctx->flash_attn_enabled, which--diffusion-fadoes not set, so a command line carried over from a FLUX setup leaves flash attention off for the text encoder without any indication. Possibly worth a note indocs/qwen_image_2.1.md.For reference, what works on this machine
--backend vulkan0 --fa --vae-tilingwith no--offload-to-cpu: the weights (16.7 GiB) go straight to VRAM, GTT is not involved at all, and 1024×1024 takes 139 s (text encode 18 s, sampling 116 s, VAE 4.5 s), peak 17.74 GiB. With a reference image, 272 s and peak 18.93 GiB.