Skip to content

feat: add Qwen Image 2.1 prefix KV cache - #2035

Merged
leejet merged 1 commit into
masterfrom
feat/qwen-image-2-1-prefix-cache
Sep 23, 2026
Merged

leejet merged 1 commit into
masterfrom
feat/qwen-image-2-1-prefix-cache

Conversation

@leejet

@leejet leejet commented Sep 23, 2026

Copy link
Copy Markdown
Owner

Summary

  • Cache text and reference-image KV during the first denoising call, then compute only target-image tokens on subsequent calls.
  • Keep separate FP32 caches for each condition within a sampling run, using existing RunnerCache lifecycle, graph cuts, and memory accounting.
  • On allocation failure, release caches and retry once without caching. Bypass caching when generation extensions are
    enabled.

Related Issue / Discussion

Fix #2031.

Additional Information

.\bin\Release\sd-cli.exe --diffusion-model  ..\models\diffusion_models\qwen_image_2.1_int8_convrot.safetensors --vae ..\models\vae\qwen_image_2.1_vae_bf16.safetensors  --llm ..\models\text_encoders\Qwen3VL-8B-Instruct-Q4_K_M.gguf  -p "a lovely cat holding a sign says 'qwen2.1.cpp'" --cfg-scale 1.0 --sampling-method euler -v --offload-to-cpu --fa

8.34it/s -> 9.58it/s

.\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\qwen_image_2.1_int8_convrot.safetensors --vae ..\models\vae\qwen_image_2.1_vae_bf16.safetensors --llm ..\models\text_encoders\Qwen3VL-8B-Instruct-Q4_K_M.gguf --llm_vision ..\models\text_encoders\Qwen3VL-8B-Instruct-mmproj-BF16.gguf -r ..\assets\qwen\qwen_image_2.1.png -p "change 'qwen2.1.cpp' to 'sd.cpp'" --cfg-scale 1.0 --sampling-method euler -v --offload-to-cpu

3.13it/s -> 5.84it/s

Checklist

@leejet
leejet merged commit 2dc7f54 into master Sep 23, 2026
9 checks passed
@leejet
leejet deleted the feat/qwen-image-2-1-prefix-cache branch September 23, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Speed is halved during qwen image 2.1 reference generation

1 participant