Skip to content

feat(unified_v11): universal 1st-order fwAD Laws 2–4 and function-space equilibrated optimizer - #587

Merged
martyn merged 3 commits into
developfrom
feat/unified-v11-universal-image-fix
Nov 20, 2026
Merged

martyn merged 3 commits into
developfrom
feat/unified-v11-universal-image-fix

Conversation

@martyn

@martyn martyn commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes unified_v11's 4D high-resolution image failure (where 0.11.0 stalled near initialization at ~5.95× worse FID than empf on CIFAR-10 Transformer-G / ResNet-18-D) using one universal, zero-autograd.grad / zero-svd / zero-eigh / zero-pinv formulation across 1D/2D vectors, deep unnormalized MLPs, conditional fibers, discrete 8×8 images, and 4D 32×32..256×256 image GANs:

  • Universal Law 2 (particlegan/_unified_v11/penalty.py): Replaces the dim <= 1024 and dim > 1024 finite-difference image secant branches (which suffered BF16 ULP underflow at img_delta=0.02, ReLU O(1/delta) kink impulses, mixed real+fake BN stat shift, and 6× over-strong logit gauge) with one universal 1st-order fwAD.dual_level() per-sample Born-Infeld critic penalty (0 torch.autograd.grad, live_endpoints_reused=True, eff_floor=0.0005/dim, eff_gauge=min(gamma_gauge, 0.005)). Temporarily evaluates _BatchNorm in eval() (leaving frozen BN untouched and restoring trainable BN flags) with a temporary BF16 dual-tangent cast hook (_bn_dual_bf16_hook, cos=+0.973 / 0.995× norm ratio vs FP32 fwAD, 7.59× faster).
  • Universal Law 3 (particlegan/_unified_v11/loss.py, critic.py): Keeps ema_margin, ema_data_scale, and ema_push_res live across all tensor ranks/dimensions without caching 4D images in _t1c_capture, and retains fibre_rel_blend=0.50 scalar logit-space relativistic coupling (softplus(f - r) / softplus(r - f)) when loss == "non_saturating" and pixel-space t1c capture is skipped (respecting explicit user loss overrides).
  • Universal Law 4 (particlegan/_unified_v11/critic.py): Removes the ndim != 2 / dim > 32 gates on _unified_v11_generator_forward_hook, using an element-budgeted slice (b_p = min(b_full, max(2, min(16, 8192 // out_dim)))) and BN-eval capture when G contains _BatchNorm.
  • Universal Function-Space Equilibrated Optimizer (particlegan/_unified_v11/optimizer.py): Removes syntactic L_count / deep_gate distortions (depth_scale, trust_mult, critic momentum) and replaces them with checkpointed envelope-tracked function-space conditioning (_ema_func_gain + generator-only stiff_gate, strictly rejecting old 0.11.0 optimizer checkpoints), degree-0 peak_rms batched Tikhonov smoothing, exact 1×1 Conv2d sqrt(out_ch / in_ch) scaling, and lifted min(r, c) <= 512 per-singular-value Tikhonov polar.
  • Tests (tests/test_unified_v11.py): Adds 4D 3×32×32 BN semantic controls & fresh-process bit-exact continuation, optimizer group-split invariance, 1×1 Conv2d parity, and strict _ema_func_gain/peak_rms tamper rejection.

Preserves all 179 Recipe fields, leaves frozen _unified/ and _unified_v4/ untouched, and preserves historical 0.11.0 Forge receipts byte for byte.

Evidence (Commit 13775ca3c, Seed 0, ZERO Recipe Overrides)

1. Real CIFAR-10 (benchmarks.cifar_toy, B=128, 1× RTX A6000, Oracle real-vs-real FID 82.66)

Horizon Recipe / Commit MoG N Final FID ↓ Best FID ↓ (step) KID×1e3 ↓ Cov /10 TV ↓ it/s ↑ Peak MiB
s800 unified_v11 (0.11.0) 200 2730.72 2730.72 (800) 66737.60 1/10 0.897 8.82 3898 (leak)
s800 bispf_optimized 200 439.36 439.36 (800) 3521.34 10/10 0.121 2.75 1485
s800 empf 200 454.21 440.22 (600) 3636.87 10/10 0.196 4.91 1496
s800 unified_v11 (13775ca3c) 200 458.95 445.31 (400) 4329.03 10/10 0.109 8.22 953
s800 unified_v11 (0.11.0) 16384 2618.93 2618.93 (800) 71734.14 2/10 0.876 8.87 3489 (leak)
s800 empf 16384 856.34 836.06 (400) 22058.32 10/10 0.276 3.92 1508
s800 unified_v11 (13775ca3c) 16384 728.10 728.10 (800) 18930.71 9/10 0.435 8.56 968
s2000 empf 200 484.64 441.55 (1200) 4738.39 10/10 0.194 4.76 1497
s2000 unified_v11 (13775ca3c) 200 493.31 436.99 (1600) 5900.29 10/10 0.172 8.49 953
s2000 empf 16384 362.57 305.10 (1600) 7951.49 10/10 0.243 4.11 1510
s2000 unified_v11 (13775ca3c) 16384 432.43 432.43 (2000) 12834.60 10/10 0.386 8.17 967

2. Official 7k 100-Gaussians Gate & Deep-16

  • 7,000-step 100-Gaussians (MoG σ=0.025, clean/live, Seed 0): PASS — 100/100 modes, hq = 99.025% (+0.50% over 0.11.0's 98.52%), mode_tv = 0.0436.
  • Deep 16-layer unnormalized MLPs (ZERO L_count): grid100_mlp16 100/100 (prec=0.9805), grid100_res16_tanh 95/100 @ 2k (climbing on 7k budget).

3. Forge discriminator_stability Tier-1 + Tier-2 Qualification (46/48 graded)

Cohort 13775ca3c 0.11.0 unified-v11 Delta
Tier 1 Required 11/11 PASS 11/11 PASS 0 (100%)
Tier 2 Required 14/16 PASS 13/16 PASS +1 NET GAIN (+2 gains: five_word_joint_hold, staggered100; -1 near-miss: model_ddgan_gaussian_chain)
Required T1 + T2 Combined 25/27 PASS 24/27 PASS +1 NET GAIN (3/3 100-mode lattices, 4/4 discrete 8×8 images, 3/3 Imbalance Nash PASS)
Tier 2 Held 6/7 PASS 7/7 PASS -1 (vector_unequal_width)
Tier 1–2 Stretch 6/10 PASS 8/10 PASS -2 (interp_sparse_arc, t3_iso_scale_mass_8x)
Total Executed T1–T2 39/46 PASS 41/46 PASS -2 stretch/held, +1 required

All 7 non-passes transparently disclosed: model_ddgan_gaussian_chain (worst_transition_cdf=0.091 vs 0.08, blocking bimodal chain prereqs model_ddgan_bimodal_chain and model_ae_ddgan_bimodal_chain), model_ae_ddgan_gaussian_chain (already FAIL on 0.11.0), vector_unequal_width (hq=0.9883, mass_tv=0.1321, sw1=0.1203 pass; 1 tail outlier cov spill), interp_sparse_arc (interp_hq=0.7536 vs 0.80), interp_img_patch_shift (already FAIL on 0.11.0), interp_saddle_barycentric (already FAIL on 0.11.0), and t3_iso_scale_mass_8x (core_cov_err=0.2972 vs 0.25).

Verification

  • python scripts/check_lint.py: PASS (294 baseline, 0 additions)
  • pytest (10 CPU suites + CUDA VRAM suite): 488 CPU passed, 3 CUDA passed, 0 failed
  • python -m experiments.forge validate && python -m experiments.forge compile --check: PASS (CURRENT, 11861/11861)

@martyn
martyn merged commit 3760b62 into develop Nov 20, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant