krea2: don't hardcode the NVIDIA-only cuDNN SDPA backend#933
Open
DasPauluteli wants to merge 1 commit into
Open
krea2: don't hardcode the NVIDIA-only cuDNN SDPA backend#933DasPauluteli wants to merge 1 commit into
DasPauluteli wants to merge 1 commit into
Conversation
The krea2 attention() forced SDPBackend.CUDNN_ATTENTION, which is NVIDIA-only. On non-NVIDIA backends (AMD ROCm, Intel XPU, Apple MPS) every forward pass fails with 'RuntimeError: No available kernel. Aborting execution.', so Krea 2 LoRA training cannot run at all there. Pass a priority list [CUDNN, FLASH, EFFICIENT, MATH] instead. NVIDIA still selects cuDNN; other backends fall back to flash/efficient/math. Verified training end-to-end on an AMD Radeon 8060S (gfx1151, ROCm 7.2).
19e2cbe to
0d6e2e4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
krea2: don't hardcode the NVIDIA-only cuDNN SDPA backend
Problem
extensions_built_in/diffusion_models/krea2/src/mmdit.pyforces the cuDNNattention backend for every attention call:
cuDNN is NVIDIA-only. On any non-NVIDIA backend (AMD ROCm, Intel XPU, Apple MPS)
there is no cuDNN kernel, so every forward pass dies immediately with:
The model loads, the VAE and Qwen3-VL text encoder run, latents/embeds cache —
and then training/inference cannot take a single step. Krea 2 is unusable on
these backends today.
Fix
Pass a priority list to
sdpa_kernelinstead of a single hardcoded backend:falls back to flash / mem-efficient / math.
One-line semantic change; no config/API surface change.
Testing
Verified Krea 2 RAW LoRA training end-to-end on an AMD Radeon 8060S
(gfx1151 / Strix Halo), ROCm 7.2, torch 2.12.1+rocm7.2:
No available kernel..safetensorsis written(
diffusion_model.*.lora_A/lora_B, loads in ComfyUI).On this GPU the fallback resolves to AOTriton flash for the unmasked image
blocks (with
TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1) and math for themasked text-fusion path. NVIDIA users are unaffected (still cuDNN).
Note for other AMD/ROCm users
This PR only fixes the hard crash. To actually run Krea 2 on ROCm you also need
TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1(for the flash path) and to have theusual audio/video imports (
torchaudio,av,librosa,mutagen) installed.Full recipe + numbers in the linked writeup.