Repository navigation

Accumulate delayed wgrads into shared (tied) parameters - #3552

Open

0z5a wants to merge 6 commits into

NVIDIA:mainfrom

0z5a:tied-delayed-wgrad-accumulate

Open

Accumulate delayed wgrads into shared (tied) parameters#3552
0z5a wants to merge 6 commits into
NVIDIA:mainfrom
0z5a:tied-delayed-wgrad-accumulate

Commits on Sep 24, 2026
- Accumulate delayed wgrads into shared (tied) parameters
  0z5a
  committed
  b5b9cc2
- Clarify the accumulate path for zero_grad(set_to_none=False)
  0z5a
  committed
  dd805bd
- Address review feedback on the tied-wgrad test file
  0z5a
  committed
  557a35b
Commits on Oct 7, 2026
- fix(pytorch): accumulate all delayed tied-parameter gradients
  0z5a
  committed
  c5cf151
- Merge main and preserve delayed tied-gradient accumulation
  0z5a
  committed
  67a310d
Commits on Oct 8, 2026
- Own delayed gradients during CUDA Graph replay
  0z5a
  committed
  dbe1326