Repository navigation
feat(unified_v11): universal 1st-order fwAD Laws 2–4 and function-space equilibrated optimizer - #587
Merged
Merged
Conversation
…, and dimension-agnostic Law 4
…timizer (Finding 1 + Finding 2)
…ery (fixes deep-16 + N=200 FID 434)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes
unified_v11's 4D high-resolution image failure (where0.11.0stalled near initialization at ~5.95× worse FID thanempfon CIFAR-10 Transformer-G / ResNet-18-D) using one universal, zero-autograd.grad/ zero-svd/ zero-eigh/ zero-pinvformulation across 1D/2D vectors, deep unnormalized MLPs, conditional fibers, discrete 8×8 images, and 4D 32×32..256×256 image GANs:particlegan/_unified_v11/penalty.py): Replaces thedim <= 1024anddim > 1024finite-difference image secant branches (which suffered BF16 ULP underflow atimg_delta=0.02, ReLUO(1/delta)kink impulses, mixed real+fake BN stat shift, and 6× over-strong logit gauge) with one universal 1st-orderfwAD.dual_level()per-sample Born-Infeld critic penalty (0torch.autograd.grad,live_endpoints_reused=True,eff_floor=0.0005/dim,eff_gauge=min(gamma_gauge, 0.005)). Temporarily evaluates_BatchNormineval()(leaving frozen BN untouched and restoring trainable BN flags) with a temporary BF16 dual-tangent cast hook (_bn_dual_bf16_hook,cos=+0.973/0.995×norm ratio vs FP32fwAD,7.59×faster).particlegan/_unified_v11/loss.py,critic.py): Keepsema_margin,ema_data_scale, andema_push_reslive across all tensor ranks/dimensions without caching 4D images in_t1c_capture, and retainsfibre_rel_blend=0.50scalar logit-space relativistic coupling (softplus(f - r)/softplus(r - f)) whenloss == "non_saturating"and pixel-spacet1ccapture is skipped (respecting explicit userlossoverrides).particlegan/_unified_v11/critic.py): Removes thendim != 2/dim > 32gates on_unified_v11_generator_forward_hook, using an element-budgeted slice (b_p = min(b_full, max(2, min(16, 8192 // out_dim)))) and BN-eval capture whenGcontains_BatchNorm.particlegan/_unified_v11/optimizer.py): Removes syntacticL_count/deep_gatedistortions (depth_scale,trust_mult, critic momentum) and replaces them with checkpointed envelope-tracked function-space conditioning (_ema_func_gain+ generator-onlystiff_gate, strictly rejecting old0.11.0optimizer checkpoints), degree-0peak_rmsbatched Tikhonov smoothing, exact1×1Conv2dsqrt(out_ch / in_ch)scaling, and liftedmin(r, c) <= 512per-singular-value Tikhonov polar.tests/test_unified_v11.py): Adds 4D3×32×32BN semantic controls & fresh-process bit-exact continuation, optimizer group-split invariance,1×1Conv2d parity, and strict_ema_func_gain/peak_rmstamper rejection.Preserves all 179
Recipefields, leaves frozen_unified/and_unified_v4/untouched, and preserves historical0.11.0Forge receipts byte for byte.Evidence (Commit
13775ca3c, Seed 0, ZERO Recipe Overrides)1. Real CIFAR-10 (
benchmarks.cifar_toy,B=128, 1× RTX A6000, Oracle real-vs-real FID82.66)s800unified_v11(0.11.0)s800bispf_optimizeds800empfs800unified_v11(13775ca3c)s800unified_v11(0.11.0)s800empfs800unified_v11(13775ca3c)s2000empfs2000unified_v11(13775ca3c)s2000empfs2000unified_v11(13775ca3c)2. Official 7k 100-Gaussians Gate & Deep-16
MoG σ=0.025, clean/live, Seed 0): PASS —100/100modes,hq = 99.025%(+0.50%over0.11.0's98.52%),mode_tv = 0.0436.L_count):grid100_mlp16100/100 (prec=0.9805),grid100_res16_tanh95/100 @ 2k (climbing on 7k budget).3. Forge
discriminator_stabilityTier-1 + Tier-2 Qualification (46/48graded)13775ca3c0.11.0unified-v11five_word_joint_hold,staggered100; -1 near-miss:model_ddgan_gaussian_chain)vector_unequal_width)interp_sparse_arc,t3_iso_scale_mass_8x)All 7 non-passes transparently disclosed:
model_ddgan_gaussian_chain(worst_transition_cdf=0.091vs0.08, blocking bimodal chain prereqsmodel_ddgan_bimodal_chainandmodel_ae_ddgan_bimodal_chain),model_ae_ddgan_gaussian_chain(already FAIL on0.11.0),vector_unequal_width(hq=0.9883, mass_tv=0.1321, sw1=0.1203pass; 1 tail outlier cov spill),interp_sparse_arc(interp_hq=0.7536vs0.80),interp_img_patch_shift(already FAIL on0.11.0),interp_saddle_barycentric(already FAIL on0.11.0), andt3_iso_scale_mass_8x(core_cov_err=0.2972vs0.25).Verification
python scripts/check_lint.py: PASS (294baseline, 0 additions)pytest(10 CPU suites + CUDA VRAM suite): 488 CPU passed, 3 CUDA passed, 0 failedpython -m experiments.forge validate && python -m experiments.forge compile --check: PASS (CURRENT,11861/11861)