Skip to content

Add Tier 2 Forge benchmark for wide conditional spread - #557

Open
mikkel wants to merge 9 commits into
developfrom
tests/forge-wide-conditional-spread
Open

mikkel wants to merge 9 commits into
developfrom
tests/forge-wide-conditional-spread

Conversation

@mikkel

@mikkel mikkel commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Registers a runnable Tier 2 stretch task in the discriminator stability view. It scores raw draws from a conditional GAN on 65 continuous features. Both groups have mean zero and exchange their narrow and wide axes. The scorer checks within-group mean and SD, coverage of 80% bag-summary intervals, interval score and energy outside the two signal coordinates. The trained-task controls include a correct sampler, a pure group swap, contracted variance, padding damage and checkpoint continuation.

The included full-budget CPU baseline uses the released EMPF configuration, seed 0 and the public deterministic initializer. G, D and the prior each received 1,200 updates, with 1,200 observed regularizer applications. Across the final generated samples, variability coverage was 65.7% and combined coverage was 71.4%. The worst group/coordinate coverage error was 23.4 percentage points against a 12-point tolerance. Forge graded the sustained result FAIL. The compact receipt and 24-frame training GIF are in reports/forge/wide-conditional-spread/.

The adapter currently supports the clean EMPF setup with a learned MoG prior and continuous penalty on each step. The check asks whether trained samples recover the right distribution. Companion PR #556 checks the observer directly.

Closes #554.

Update (base: develop). Merged current develop. Develop's discriminator_stability revisions 17–19 are kept byte-for-byte; revision 19 is archived and this task is added as revision 20 (Tier 2 stretch, order 107). The unmerged 2026-11-17 fixed test list is replaced by technique-inventory-full-suite-2026-11-20.json (develop's 69-test 2026-11-18 list + this task = 70). Because the host binds clean EMPF only, the SigReg admission test records it as an explicit, checked exclusion (preflight must still report BLOCKED). The word-hold budget test keeps the historical 76,620 s total and declares this task's own 600 s reservation separately. Catalog and Forge reports regenerated with the repo's tooling.

🤖 Generated with Claude Code

mikkel and others added 9 commits November 17, 2026 23:40
Keep develop's discriminator_stability revisions 17-19 byte-for-byte, archive
revision 19, and add wide_conditional_spread as revision 20 (Tier 2 stretch).
Replace the unmerged 2026-11-17 fixed test list with a 2026-11-20 list that
extends develop's 2026-11-18 list by this one task. Generated forge reports
take develop's version and are regenerated in the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ls explicitly

The wide conditional spread stretch host binds clean EMPF only and reports
BLOCKED for other recipes; record that as a checked exclusion in the SigReg
admission test. Keep the historical revision-19 timeout total (76620 s) and
declare the new task's own 600 s reservation separately. Refresh the Forge
catalog with the history importer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Take develop's catalog and refresh it with the Forge history importer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rebuilt with the repo's refresh tooling (refresh_compiled_reports): technique
inventory, family pages, experiments-by-tier, test performance, compiled memory.
No training; published qualification evidence preserved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n 20

test_r3_calibration_holdout pinned the current revision at 19; the T3 isolation
test assumed revision 18 was the immediate predecessor. Walk the archived chain
instead. Archived revisions are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant