test: harden external DMG validation and gate dmg-acid2

.github/workflows/external-roms.yml

@@ -7,7 +7,7 @@ on:
   workflow_dispatch:
 
 jobs:
-  blargg-dmg:
+  dmg-gate:
     runs-on: ubuntu-latest
 
     steps:
@@ -29,5 +29,5 @@ jobs:
       - name: Fetch external ROMs
         run: make fetch-external-roms
 
-      - name: Run supported external Blargg DMG suites
-        run: make test-external-blargg-dmg
+      - name: Run supported external DMG suites
+        run: make test-external-dmg

.gitignore

@@ -11,6 +11,7 @@
 
 # Logs
 *.log
+/.artifacts/
 
 # Local ROM stores
 /.roms/

AI/ROADMAP.md

@@ -1805,6 +1805,29 @@ Phase `6` cartridge persistence is intentionally not a substitute for this whole
 This phase is the roadmap home for the final DMG closure work. Parts of it should begin earlier, but the block only closes once the project can justify DMG correctness through layered evidence on the shared T-cycle model rather than through informal game compatibility.
 It assumes the dedicated save-state and serialization infrastructure from Phase 8 already exists and uses it as part of closure evidence.
 
+Status note (`2026-03-22`): the repo now starts a narrow early hardening lane
+from this phase before APU work. The current early deliverables are:
+
+- one explicit partial subsystem checklist in `AI/TESTING.md` that distinguishes
+  repo-gated external evidence from internal-only evidence for the already-landed
+  DMG subsystems
+- one `gb-test-runner` catalog path,
+  `cargo run -p gb-test-runner --bin run_rom_suite -- --list-detailed`, that
+  exposes the built-in suite set together with oracle channel, capture, and
+  retained-artifact policy
+- one `gb-test-runner` checklist path,
+  `cargo run -p gb-test-runner --bin run_rom_suite -- --early-checklist`, that
+  exposes the current early hardening status per subsystem together with the
+  evidence already landed and the still-open closure gaps
+- one repo-gated PPU framebuffer-oracle suite,
+  `cargo run -p gb-test-runner --bin run_rom_suite -- --suite gbdev-dmg-acid2`,
+  sourced from `GBEmulatorShootout` and now part of the supported external DMG
+  block used by `make local` and the `external-roms` workflow
+
+This does not count as closing Phase `9.2` or `9.3`; SameBoy/Gambatte
+differential tooling, save/load determinism, and the final DMG matrix still
+remain Phase `7/8/9` work.
+
 #### Goal
 
 Close the DMG core with a formal validation matrix, strong differential and determinism tooling, and explicit closure criteria that leave no major blind hardware areas behind.
@@ -1968,7 +1991,7 @@ Suggested entry style:
 - [PPU][SKIPBOOT-ORACLE] The Phase `4.1` `SkipBoot` startup-mode latch is still validated only against repo-local continuity tests and the documented post-boot snapshot contract. Before Phase `9` hardening treats the direct-boot PPU handoff as externally validated, this path still needs comparison against a trusted oracle or hardware-derived capture proving that the first LCD-visible dots after `SkipBoot` remain coherent with the published `LCDC`, `STAT`, and `LY` state rather than with an overfit local latch assumption. Phase dependency: this does not block Phase `5`, but later DMG closure should not claim externally validated direct-boot PPU continuity until this check lands.
 - [PPU][WINDOW-GLITCH-ORACLE] The current Phase `4.4` window baseline includes explicit tested paths for `WX = 0` and `WX = 166`, but they remain provisional baseline behavior rather than oracle-backed glitch closure. The project still needs stricter validation and, if necessary, refinement for `WX` / `WY` / `LCDC.5` mid-frame glitch behavior, including the DMG-specific `WX = 0 && (SCX & 7) > 0` path and the special `WX = 166` continuation behavior. Phase dependency: this does not block entering Phase `5`, but Phase `9` hardening should not mark detailed DMG window-glitch behavior as closed until this oracle pass is finished.
 - [PPU][LCDC2-8X16-ARTIFACTS] The Phase `4.5` sprite baseline already treats `LCDC.2` as live state and covers the core `8x16` row-selection rules, but the finer DMG-visible artifacts and leaks caused by mid-frame `LCDC.2` size changes, especially around the lower half of `8x16` sprites, remain only documented follow-up work. Before Phase `9` hardening claims detailed sprite-size behavior as externally validated, this path still needs targeted ROM or oracle coverage and, if needed, refinement that keeps those artifacts explicit instead of leaving them as accidental baseline behavior. Phase dependency: this does not block Phase `5`, but later DMG closure should not claim fully hardened `LCDC.2` / `8x16` edge behavior until this validation lands.
-- [PPU][OAM-CORRUPTION-ORACLE] Phase `4.8` now has deterministic unit/integration coverage, a shipped synthetic ROM/trace family for direct Mode `2` OAM access, `FEA0-FEFF` reads, `inc rr`, `[hli]` / `[hld]`, stack plus interrupt-service paths, DMG-family model variants, and the CGB negative path, but it still lacks comparison against an independent trusted oracle or hardware-derived capture before the bug can be treated as externally validated across instruction families and hardware revisions. The current strict-mode official `retrio/blargg oam_bug` bring-up narrowed the remaining live gap to the detailed timing/effect closure path: `1-lcd_sync`, `4-scanline_timing`, and `5-timing_bug` are green with the current restart plus Mode `2` row baseline, while `7-timing_effect` still diverges and should stay the active external-validation target before Phase `9` claims OAM corruption closure. Phase dependency: this does not block moving into later PPU or APU work, but Phase `9` hardening should not mark OAM corruption as fully closed until that independent-validation pass is done.
+- [PPU][OAM-CORRUPTION-ORACLE] Phase `4.8` now has deterministic unit/integration coverage, a shipped synthetic ROM/trace family for direct Mode `2` OAM access, `FEA0-FEFF` reads, `inc rr`, `[hli]` / `[hld]`, stack plus interrupt-service paths, DMG-family model variants, and the CGB negative path, but it still lacks comparison against an independent trusted oracle or hardware-derived capture before the bug can be treated as externally validated across instruction families and hardware revisions. As of March 21, 2026, the built-in official `retrio/blargg oam_bug` automation is intentionally curated to the `GBEmulatorShootout`-listed subset, the repository-gated external DMG block now runs that curated subset by default, and the automation still excludes the upstream multi-ROM `oam_bug.gb` plus `7-timing_effect.gb`; Phase `9` hardening should not treat that curated subset as complete OAM-corruption closure without a later independent oracle pass for the remaining timing-sensitive coverage.
 
 #### Done:
 
@@ -1999,7 +2022,7 @@ Suggested entry style:
 
 ### Phase 9 — Final DMG hardening, differential validation, and closure
 
-- [TEST-RUNNER][OFFICIAL-CGB-INTERRUPT-TIME] `retrio/blargg interrupt_time` is now wired with the correct `ConsoleModel::Cgb` metadata, but it still does not pass because the core lacks the CGB CPU-speed behavior that the upstream ROM explicitly exercises. Phase dependency: this does not block current DMG hardening or Phase `7`, but it should stay visible as future CGB bring-up work instead of being misread as a DMG regression.
+- None currently.
 
 #### Done:
 

AI/TESTING.md

@@ -43,13 +43,61 @@ Every subsystem change should aim to leave behind one of these:
 - For this T-cycle-based core, the matrix should support several validation granularities: end of test, end of instruction, and short per-T-cycle windows when a timing-sensitive divergence needs to be isolated.
 - Every corrected bug should end up attached to at least one permanent layer in this matrix.
 
+## Early Phase 9 partial checklist
+
+Before full Phase `7/8/9` closure exists, keep one explicit partial-hardening
+checklist for the already landed subsystems. This checklist is not final DMG
+closure; it is the minimum "do not keep flying blind" matrix used while the
+project is still bringing up later hardware blocks such as APU and save states.
+
+- `CPU`: repo gate present. Current evidence: Phase `2` synthetic timing ROMs plus
+  external Blargg `cpu_instrs` smoke / full and `instr_timing`. Remaining final-closure
+  gaps: differential oracle workflow, replay / determinism, and broader boot-path
+  arbitration.
+- `Interrupts`: repo gate present. Current evidence: Phase `2` interrupt timing ROMs,
+  Blargg `halt_bug`, and the interrupt-heavy `cpu_instrs` cases. Remaining final-closure
+  gaps: dedicated differential traces and longer-running determinism coverage.
+- `Timer`: internal gate only. Current evidence: Phase `2` timer and interrupt-service
+  synthetic ROMs plus unit coverage in `gb-core`. Remaining final-closure gaps:
+  dedicated external timing suites and differential oracle coverage.
+- `Bus`: repo gate present. Current evidence: bus/DMA/CPU integration coverage plus
+  external Blargg `mem_timing`, `mem_timing-2`, and their individual ROMs. Remaining
+  final-closure gaps: broader cartridge-family oracle comparisons and save/load-era
+  determinism checks.
+- `DMA`: internal gate only. Current evidence: closed Phase `3` unit and integration
+  coverage. Remaining final-closure gaps: a promoted external ROM slice and later
+  differential tooling.
+- `PPU`: repo gate present for the currently closed OAM-corruption slice. Current
+  evidence: Phase `4` synthetic OAM-corruption ROMs, curated Blargg `oam_bug`
+  singles `1..6,8`, and a repo-gated `dmg-acid2` framebuffer oracle.
+  Remaining final-closure gaps: mealybug tearoom, broader rendering/timing
+  differential coverage, and the still-deferred non-curated exploratory ROMs.
+- `Cartridge`: internal gate only. Current evidence: unit and integration coverage
+  for `NoMbc`, `Mbc1`, `Mbc2`, `Mbc3`, `Mbc5`, and hardware-style persistence.
+  Remaining final-closure gaps: external oracle material beyond synthetic/unit
+  coverage and later save/load determinism once Phase `8` exists.
+- `Joypad` / `Serial`: internal gate only. Current evidence: closed Phase `5`
+  synthetic coverage and subsystem tests. Remaining final-closure gaps: promoted
+  external closure ROMs and differential comparison hooks.
+
+This checklist should move only when one of the following becomes true:
+
+- a subsystem gains a repo-gated external suite with explicit pass/fail policy
+- a subsystem gains first-divergence differential tooling against a trusted oracle
+- a previously internal-only block acquires deterministic replay or save/load
+  continuation evidence
+- a discovered regression forces the checklist back from "repo gate present" to
+  "internal gate only" until the external evidence is green again
+
 ## External ROM harness policy
 
 - Test ROM execution must be automatable; manual inspection of the LCD is an auxiliary debugging aid, not the primary acceptance path.
 - The harness should support at least framebuffer capture and serial / link-port capture when the ROM exposes machine-readable output there.
 - When an external suite prints self-validating text through a documented screen-console protocol, the harness should also support a typed text-extraction path for that protocol rather than falling back to a circular framebuffer fixture generated by this project.
 - Prefer serial / link-port capture for suites such as Blargg `cpu_instrs` when that path is available, because it avoids treating a scrolling framebuffer as the primary machine-readable result channel.
 - For the current official Blargg bring-up, the harness now supports three machine-readable channels: serial, cartridge-RAM text/status output, and the upstream `console.s` BG-map text console used by screen-driven ROMs such as `halt_bug`.
+- Before promoting a third-party ROM into the built-in automated catalog, verify whether it is listed in `GBEmulatorShootout` and record any explicit exception instead of assuming the ROM belongs in the default curated set.
+- A ROM omitted from `GBEmulatorShootout` may still be useful for exploratory debugging, but it must stay out of the repo-managed built-in suites until the reason for including it anyway is documented explicitly.
 - Each ROM case should define a timeout, an explicit pass/fail rule, and retained failure artifacts such as serial output, framebuffer output, trace excerpts, and optional snapshots.
 - When a ROM needs deterministic host-side interaction, the typed case metadata should also carry the external stimulus schedule explicitly instead of burying that behavior in ad hoc test-only closures.
 - During early Phase `0`, `gb-test-runner` could begin as a contract-only crate, but it should already own typed ROM-case and suite metadata including console model, startup mode, execution mode, emulation-progress timeout, explicit pass/fail rule, external stimulus schedule when needed, requested captures, and retained failure-artifact policy.
@@ -60,6 +108,16 @@ Every subsystem change should aim to leave behind one of these:
 - Keep redistributable external test ROMs and non-redistributable commercial ROMs in separate stores. The current local-only commercial bucket is `/.roms/local-commercial/`, and it must remain outside CI, docs about official closure, and public automation targets.
 - The minimum DMG closure baseline should include automated CPU / interrupt coverage through `retrio/gb-test-roms` or equivalent Blargg automation, `dmg-acid2` for basic DMG PPU validation, and `mealybug-tearoom-tests` for fine PPU rendering / timing validation.
 - Keep explicit roadmap space for broader closure suites such as Mooneye / Gekkio coverage, SameSuite, GB Accuracy Tests, 144p Test Suite, and MBC3 RTC-focused ROMs.
+- `gb-test-runner` should expose a human-readable catalog of the built-in suites
+  and their active oracle channels. The current CLI entry point is
+  `cargo run -p gb-test-runner --bin run_rom_suite -- --list-detailed`.
+- The same harness should also expose the current early hardening checklist so
+  the repo can answer "what is externally gated already and what is still
+  internal only?" without re-reading the docs. The current CLI entry point is
+  `cargo run -p gb-test-runner --bin run_rom_suite -- --early-checklist`.
+- The current early PPU hardening lane also includes the repo-gated framebuffer
+  oracle suite for `dmg-acid2` under
+  `cargo run -p gb-test-runner --bin run_rom_suite -- --suite gbdev-dmg-acid2`.
 
 ## Differential oracle policy
 
@@ -95,7 +153,12 @@ Every subsystem change should aim to leave behind one of these:
 - Before opening or updating a pull request, run at least `make check` locally so formatting, clippy, tests, typos, and `cargo deny` do not first surface in CI.
 - When a change touches CI, coverage, dependency policy, repo tooling, or other workflow-critical infrastructure, run `make local` locally as well before the PR is updated.
 - The GitHub `ci` workflow is intentionally limited to formatting, linting, tests, typos, dependency policy, and the coverage gate. Keep external ROM execution out of that workflow.
-- The repository-gated external official ROM block currently includes the green non-APU, non-CGB Blargg DMG suites (`cpu_instrs` smoke plus full multi-ROM, `instr_timing`, `halt_bug`, `mem_timing`, and `mem_timing` individual ROMs). Keep `oam_bug`, `interrupt_time`, and APU-specific suites out of the default external-ROM workflow until they are green and intentionally promoted.
+- The repository-gated external official ROM block currently includes the green
+  non-APU, non-CGB DMG suites: the Blargg DMG block (`cpu_instrs` smoke plus
+  full multi-ROM, `instr_timing`, `halt_bug`, `mem_timing`, `mem_timing`
+  individual ROMs, and the curated `oam_bug` singles `1..6` plus `8`) plus the
+  `dmg-acid2` framebuffer oracle. Keep APU-specific suites out of the default
+  external-ROM workflow until they are green and intentionally promoted.
 - For the current infrastructure-heavy stage, keep the aggregate coverage gate across `gb-core`, `gb-test-runner`, and `gb-persistence` at or above `90%` for lines, regions, and functions, but do not satisfy that threshold with hollow tests that only exercise trivial getters or app placeholders.
 - When immediate automated coverage is temporarily impractical, record the missing test coverage, the reason it is deferred, and the remaining risk in the change report; add a roadmap TODO as well if the gap is concrete and non-trivial.
 - ROM-based validation and oracle comparison complement automated tests; they do not replace the expectation that new code should usually leave behind unit or integration coverage.

Makefile

@@ -1,16 +1,18 @@
 .DEFAULT_GOAL := check
 
-.PHONY: help setup hooks tools check local fetch-external-roms test-external-blargg-dmg
+.PHONY: help setup hooks tools check local fetch-external-roms test-external-dmg test-external-blargg-dmg test-external-ppu-dmg
 
 help:
 	@echo "Available targets:"
 	@echo "  make setup                Configure git hooks and install local cargo tools"
 	@echo "  make hooks                Configure repository git hooks"
 	@echo "  make tools                Install local cargo tools used by this repository"
 	@echo "  make check                Run the local pre-push gate (fmt, clippy, test, typos, deny)"
-	@echo "  make local                Run check plus external Blargg DMG suites and coverage"
+	@echo "  make local                Run check plus the supported external DMG suites and coverage"
 	@echo "  make fetch-external-roms  Populate .roms/external-test from the pinned manifest"
+	@echo "  make test-external-dmg    Run the supported external DMG block"
 	@echo "  make test-external-blargg-dmg  Run the supported external Blargg DMG block"
+	@echo "  make test-external-ppu-dmg     Run the supported external DMG PPU block"
 
 setup: hooks tools
 
@@ -33,17 +35,25 @@ check:
 
 local: check
 	$(MAKE) fetch-external-roms
-	$(MAKE) test-external-blargg-dmg
+	$(MAKE) test-external-dmg
 	cargo cov-check
 	cargo cov-lcov
 
 fetch-external-roms:
 	cargo run -q -p gb-test-runner --bin fetch_external_roms --
 
+test-external-dmg:
+	$(MAKE) test-external-blargg-dmg
+	$(MAKE) test-external-ppu-dmg
+
 test-external-blargg-dmg:
 	cargo test --release -p gb-test-runner --test external retrio_blargg_cpu_smoke_suite_runs_against_real_external_assets -- --ignored --exact --nocapture
 	cargo test --release -p gb-test-runner --test external retrio_blargg_cpu_instrs_full_suite_runs_against_real_external_assets -- --ignored --exact --nocapture
 	cargo test --release -p gb-test-runner --test external retrio_blargg_instr_timing_suite_runs_against_real_external_assets -- --ignored --exact --nocapture
 	cargo test --release -p gb-test-runner --test external retrio_blargg_halt_bug_suite_runs_against_real_external_assets -- --ignored --exact --nocapture
 	cargo test --release -p gb-test-runner --test external retrio_blargg_mem_timing_suite_runs_against_real_external_assets -- --ignored --exact --nocapture
 	cargo test --release -p gb-test-runner --test external retrio_blargg_mem_timing_individual_suite_runs_against_real_external_assets -- --ignored --exact --nocapture
+	cargo test --release -p gb-test-runner --test external retrio_blargg_oam_bug_suite_runs_against_real_external_assets -- --ignored --exact --nocapture
+
+test-external-ppu-dmg:
+	cargo test --release -p gb-test-runner --test external gbdev_dmg_acid2_suite_runs_against_real_external_assets -- --ignored --exact --nocapture