Skip to content

Fix CUDA OOM during Krea2 block-by-block quantization#927

Open
thedivergentai wants to merge 1 commit into
ostris:mainfrom
thedivergentai:fix/krea2-quantize-oom
Open

Fix CUDA OOM during Krea2 block-by-block quantization#927
thedivergentai wants to merge 1 commit into
ostris:mainfrom
thedivergentai:fix/krea2-quantize-oom

Conversation

@thedivergentai

Copy link
Copy Markdown

Summary

  • Fix CUDA OOM during Krea2 transformer quantization by synchronizing and flushing GPU memory after each block is offloaded to CPU.
  • Ensure Krea2 starts quantization from a clean CPU baseline, including assistant LoRA weights when present.

Test plan

  • Reproduced original failure path (OOM around block 25/28 during quantize_model)
  • Verified all 28 blocks quantize successfully on a 24GB RTX 3090
  • Confirmed Krea2 Raw training job completes model load and starts training

Synchronize and flush GPU memory after each transformer block offload so async quanto copies cannot accumulate VRAM across all 28 layers.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant