fix(ci): enforce compile-fail diagnostic contract

[agourakis82]

Summary

• Enforce each declared //@ error-pattern in the v2 compile-fail runner.
• Reject timeouts, harness failures, and signal termination instead of treating them as expected compiler rejection.
• Add a deterministic CI contract gate for matching diagnostics, wrong diagnostics, SIGSEGV, and timeout.
• Classify the 27 stage2-native diagnostic mismatches exposed by the new contract as named full-suite xfails.

Fixes #444.

Root Cause

The v2 runner's error-pattern extraction regex left the pattern list empty. It also reset every non-timeout nonzero compile status to success, so an exit 139 SIGSEGV could satisfy a compile-fail fixture even when no compiler diagnostic had been validated.

Validation

• bash scripts/ci/compile_fail_harness_contract_gate.sh
• bash scripts/dev/check_workflow_script_refs.sh
• bash -n scripts/dev/run_sio_test_suite_v2.sh
• bash -n scripts/ci/compile_fail_harness_contract_gate.sh
• python3 -c 'import yaml; yaml.safe_load(open(".github/workflows/ci.yml", encoding="utf-8")); print("ci.yml parsed")'
• SOUNIO_TEST_JOBS=1 bash scripts/run_sio_test_suite.sh --filter-exact parser_rust_let_mut.sio --verbose --jobs 1
• Synthetic JUnit manifest-load check with a temporary rejecting wrapper: 101 known failures and 0 failures
• git diff --check

Residual Diagnostic Backlog

The first remote full-suite run exposed 27 compile-fail/UI fixtures whose stage2-native compiler output does not contain their declared diagnostic text. They are now explicit xfails in tests/known_failures/hardened_diagnostics_full_suite.txt; the harness remains strict and must not be weakened to make them pass.

Blocker Record

Blocker-ID: BLK-20260717-compile-fail-harness-contract
Status: merge-ready
Severity: B1
Class: harness-routing
Owner: Codex
Lane: compile-fail-harness-contract
Worktree: /tmp/sounio-compile-fail-contract-20260717
Branch: codex/compile-fail-contract-20260717
Files-Owned: scripts/dev/run_sio_test_suite_v2.sh, scripts/ci/compile_fail_harness_contract_gate.sh, .github/workflows/ci.yml, tests/known_failures/hardened_diagnostics_full_suite.txt
Files-Read-Only: scripts/lib/resolve_souc.sh, tests/compile-fail/parser_rust_let_mut.sio
Do-Not-Touch: scripts/run_sio_test_suite.sh, scripts/lib/resolve_souc.sh
Repro: bash scripts/ci/compile_fail_harness_contract_gate.sh
Observed: a matching signal exit and an error-pattern mismatch could both be accepted as compile-fail passes
Expected: only a diagnostic rejection matching every declared error-pattern may pass; timeout, harness exit, and signal termination fail
Acceptance-Gate: bash scripts/ci/compile_fail_harness_contract_gate.sh
Evidence-Level: E4
Evidence: first remote Full Test Suite classified 27 stale diagnostic-pattern mismatches; https://github.com/Sounio-lang/sounio/actions/runs/29554999248 confirms Contracts, Full Test Suite, and CI Decision green
Fallback-Path: none
Legacy-Kept: yes; the v1 runner is intentionally untouched
LLM-Offload: not-required
Next-Action: human review and merge decision; keep the PR draft until promoted explicitly

Blocker-ID: BLK-20260717-stage2-diagnostic-pattern-backlog
Status: classified
Severity: B1
Class: compiler-semantics
Owner: Codex
Lane: stage2-diagnostic-contract-triage
Worktree: /tmp/sounio-compile-fail-contract-20260717
Branch: codex/compile-fail-contract-20260717
Files-Owned: tests/known_failures/hardened_diagnostics_full_suite.txt
Files-Read-Only: affected tests/compile-fail and tests/ui fixtures
Do-Not-Touch: compiler semantic sources in this harness lane
Repro: remote Full Test Suite run 29554394979
Observed: 27 fixtures reject compilation but miss their declared error-pattern under the stage2-native artifact
Expected: each fixture emits every declared pattern and can be removed from the xfail manifest
Acceptance-Gate: Full Test Suite with no corresponding manifest entry
Evidence-Level: E4
Evidence: https://github.com/Sounio-lang/sounio/actions/runs/29554394979 exposed the 27 mismatches; https://github.com/Sounio-lang/sounio/actions/runs/29554999248 confirms their explicit xfail classification
Fallback-Path: audited xfail manifest; no silent pass
Legacy-Kept: n/a
LLM-Offload: not-required
Next-Action: split the diagnostics by compiler component and remove xfails only with matching stage2-native receipts

[github-actions]

CI Watch: GREEN

Run: 29554999248
Head: b8973777ba4f
Failed jobs: none

Next action: All selected checks passed; ready for merge subject to repository policy.

This watcher observes CI only. It cannot merge, edit code, or decide scientific semantics.