Skip to content

fix(critic): break autograd hook reference cycle leaking VRAM in _live_captures - #585

Merged
martyn merged 1 commit into
developfrom
fix/unified-critic-vram-leak
Nov 20, 2026
Merged

martyn merged 1 commit into
developfrom
fix/unified-critic-vram-leak

Conversation

@martyn

@martyn martyn commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes a C++/Python reference-cycle VRAM leak in ParticleGAN's critic _live_captures forward hooks. Previously, capturing entry inside unscaled.register_hook(lambda _g, _e=entry: _e.__setitem__(5, False)) formed a cycle (unscaled -> AutogradMeta hook -> entry -> unscaled + x_act) that leaked 2 input batches (and their autograd graphs) per step even after _live_captures.clear().

Fixed sites (all 6 cycle sites across particlegan/)

  • particlegan/_unified_v11/critic.py
  • particlegan/_unified_v4/critic.py
  • particlegan/_unified/critic.py
  • particlegan/_t2_critic/critic.py (added shared _clear_live_capture_entries / _trim_live_captures helpers)
  • particlegan/_nash_v3/record.py (also added record_step() cleanup on _NashV3ObserverStepRecord)
  • particlegan/_minimal/dynamics.py (inline standalone fix keeping minimal core < 900 LOC)

Mechanism

  • Store the backward-alive flag in a separate 1-element list (alive = [True]) and hook raw_target/target with lambda _g, _a=alive: _a.clear() so bool(alive) flips True -> False idempotently across all 4 existing consumers (_unified/penalty.py, _t2_critic/penalty.py, _nash_v3/penalty.py, _minimal/geometry.py) without modifying any loss/penalty/optimizer formulas, and properly invalidates alive on BF16 separate-backward passes.
  • Explicitly null entry[3] = entry[4] = None and clear entry[5] in clear_transient_captures(), ring-buffer eviction, and checkpoint restore (load_state_dict / _restore_*_packet).

Testing

Added tests/test_unified_critic_vram_leak.py covering weakref GC (fp32 & bf16), ring-buffer eviction, load_state_dict release, backward-alive semantic controls across all 4 consumers, CPU multi-step GANTrainer zero-retained-batch checks, and CUDA 40-step CIFAR-10-sized flat-VRAM checks.

  • python3 scripts/check_lint.py: PASS (294 existing findings; no additions)
  • CPU pytest (test_unified_critic_vram_leak.py, test_unified_v11.py, test_unified_v4.py, test_unified_recipe.py, test_t2_secant_critic.py, test_nash_v3.py, test_minimal.py, test_training.py, test_recipe_defaults.py): 177 passed, 0 failed
  • CUDA_VISIBLE_DEVICES=0 python3 -m pytest -q -m cuda tests/test_unified_critic_vram_leak.py: 3 passed, 0 failed (0 leaked batch weakrefs, flat VRAM over 40 steps on unified_v11, unified_v4, unified)
  • python3 -m experiments.forge validate && python3 -m experiments.forge compile --check: PASS (valid, CURRENT)

@martyn
martyn merged commit fb1fd32 into develop Nov 20, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant