Skip to content

krea2: don't hardcode the NVIDIA-only cuDNN SDPA backend#933

Open
DasPauluteli wants to merge 1 commit into
ostris:mainfrom
DasPauluteli:fix/krea2-sdpa-backend-portability
Open

krea2: don't hardcode the NVIDIA-only cuDNN SDPA backend#933
DasPauluteli wants to merge 1 commit into
ostris:mainfrom
DasPauluteli:fix/krea2-sdpa-backend-portability

Conversation

@DasPauluteli

Copy link
Copy Markdown

krea2: don't hardcode the NVIDIA-only cuDNN SDPA backend

Problem

extensions_built_in/diffusion_models/krea2/src/mmdit.py forces the cuDNN
attention backend for every attention call:

with sdpa_kernel(SDPBackend.CUDNN_ATTENTION):
    x = F.scaled_dot_product_attention(q, k, v, attn_mask=mask, scale=scale, enable_gqa=gqa)

cuDNN is NVIDIA-only. On any non-NVIDIA backend (AMD ROCm, Intel XPU, Apple MPS)
there is no cuDNN kernel, so every forward pass dies immediately with:

RuntimeError: No available kernel. Aborting execution.
  File ".../krea2/src/mmdit.py", line 60, in attention
    x = F.scaled_dot_product_attention(...)

The model loads, the VAE and Qwen3-VL text encoder run, latents/embeds cache —
and then training/inference cannot take a single step. Krea 2 is unusable on
these backends today.

Fix

Pass a priority list to sdpa_kernel instead of a single hardcoded backend:

with sdpa_kernel([
    SDPBackend.CUDNN_ATTENTION,
    SDPBackend.FLASH_ATTENTION,
    SDPBackend.EFFICIENT_ATTENTION,
    SDPBackend.MATH,
]):
    x = F.scaled_dot_product_attention(...)
  • NVIDIA: behaviour is unchanged — cuDNN is still first and gets selected.
  • Everything else: the dispatcher skips the unavailable cuDNN kernel and
    falls back to flash / mem-efficient / math.

One-line semantic change; no config/API surface change.

Testing

Verified Krea 2 RAW LoRA training end-to-end on an AMD Radeon 8060S
(gfx1151 / Strix Halo), ROCm 7.2, torch 2.12.1+rocm7.2
:

  • Before: crashes on the first step with No available kernel.
  • After: all training steps run and a valid LoRA .safetensors is written
    (diffusion_model.*.lora_A/lora_B, loads in ComfyUI).

On this GPU the fallback resolves to AOTriton flash for the unmasked image
blocks (with TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1) and math for the
masked text-fusion path. NVIDIA users are unaffected (still cuDNN).

Note for other AMD/ROCm users

This PR only fixes the hard crash. To actually run Krea 2 on ROCm you also need
TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 (for the flash path) and to have the
usual audio/video imports (torchaudio, av, librosa, mutagen) installed.
Full recipe + numbers in the linked writeup.

The krea2 attention() forced SDPBackend.CUDNN_ATTENTION, which is
NVIDIA-only. On non-NVIDIA backends (AMD ROCm, Intel XPU, Apple MPS)
every forward pass fails with 'RuntimeError: No available kernel.
Aborting execution.', so Krea 2 LoRA training cannot run at all there.

Pass a priority list [CUDNN, FLASH, EFFICIENT, MATH] instead. NVIDIA
still selects cuDNN; other backends fall back to flash/efficient/math.
Verified training end-to-end on an AMD Radeon 8060S (gfx1151, ROCm 7.2).
@DasPauluteli DasPauluteli force-pushed the fix/krea2-sdpa-backend-portability branch from 19e2cbe to 0d6e2e4 Compare July 4, 2026 21:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant