fix(gate): hollow-results guard + bound the execution outer loop (both crossings were fake)

STATUS AFTER A DAY: 449 projects at in_progress, 2 "crossed" — and BOTH crossings
were hollow. The gate let them through and the fleet was starving.

1. HOLLOW RESULTS (execution/hollow_guard.py — new deterministic gate)
   The gate's bar was `bool(produced)`: "did a file appear?". It NEVER looked inside.
   PROJ-179 ("The Influence of Metacognitive Awareness") reached research_complete
   having run on the IRIS FLOWER dataset (150 rows, 105/45 split) and written:
       primary_analysis.json    {"correlation_coefficient": null, "p_value": null}
       correlation_metrics.json {"d_prime": NaN}
       robustness_analysis.json []
       validation_report.json   {"status": "PASS"}   <- self-certified
   Its confidence_rating column — the study's KEY measure — was blank in all 150 rows.
   fabrication_guard catches numbers that were FAKED; this catches numbers that were
   never COMPUTED. Both mean "not a real measurement" and both now hard-fail the gate.
   Tuned for HIGH PRECISION: 0.0 and negatives are real values and are never flagged.

2. NO DURABLE EVIDENCE (same module)
   PROJ-256 advanced to research_review on ONE artifact —
   data/processed/null_fpr_metrics.json — which is GITIGNORED. Nothing was committed:
   no result a reviewer can open, nothing a paper can cite. Its entire empirical
   contribution had already evaporated. A run whose every artifact is gitignored now
   fails. Durability is decided by git itself (the .gitignore stays the SSoT).

   Both projects are reset to in_progress with the truth recorded. Honest count: 0.

3. UNBOUNDED OUTER LOOP (state/execution_status.py + graph.py)
   reset_fix_loop wiped BOTH fix_rounds and model_tier, so a project whose analysis
   cannot run climbed the whole ladder (12 rounds x every tier), re-planned to a
   clean slate, and climbed it AGAIN — forever, with no memory. Two monotonic
   counters now survive the resets: `total_attempts` and `replan_rounds`. Past
   MAX_REPLAN_ROUNDS the honest outcome is terminal VALIDATOR_REJECTED — the
   analysis is not executable here — not another lap. (Still autonomous; never
   human_input_needed.)

4. CHURNERS MONOPOLISED THE MATRIX (scheduler.py)
   A failed execution re-opens only a handful of tasks, so a hopeless project sits at
   ~6 remaining FOREVER and shortest-remaining-first re-picked it every tick.
   PROJ-029 took 843 implementer calls in ONE day; 12 churners ate ~4,000 of the
   day's 5,111 while 400 projects got NONE. Distance-from-done now includes
   EXEC_CHURN_PENALTY * total_attempts, so churn falls behind healthy work without
   being banished (it still needs attempts to reach the re-plan cap). Live effect:
   the in_progress workers now pick healthy projects instead of the six churners.

   scripts/backfill_execution_churn_counters.py replays git history to seed both
   counters on the 36 existing records (PROJ-586: 38 attempts; PROJ-843: 33/2 replans)
   — without it, both new guards would have been blind to the churn already burned.

5. FLAPPING TEST (tests/unit/test_data_contract.py)
   It asserted against LIVE state (state/execution_status/PROJ-262.json + the live
   project tree), which the pipeline rewrites every tick. The evidence moved out from
   under it and it failed while the detector it guards was fine. The REAL evidence
   (404a05f) is now frozen in tests/fixtures/proj262/ — verbatim, assertions
   unchanged, no longer a moving target.

Full offline suite: 6436 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: github-actions[bot]

ruff.toml

@@ -11,6 +11,11 @@ src = ["src", "tests"]
 # config and we can't run the project-wide lint gate.
 extend-exclude = [
   "projects/",
+  # Frozen REAL project code captured as regression evidence (e.g.
+  # tests/fixtures/proj262/). It is agent-generated project-side code preserved
+  # VERBATIM — linting or reformatting it would corrupt the very evidence the
+  # test asserts against. Same rationale as `projects/` above.
+  "tests/fixtures/",
   ".venv/",
   "build/",
   "dist/",

scripts/backfill_execution_churn_counters.py

@@ -0,0 +1,95 @@
+"""One-time backfill of the monotonic execution-churn counters from git history.
+
+``total_attempts`` and ``replan_rounds`` were added after the fact, so every existing
+``state/execution_status/*.json`` reads 0 — which would blind BOTH new guards to the
+churn that already happened: the scheduler's anti-monopoly penalty would not fire, and
+a project that has already been round the ladder several times would be granted
+MAX_REPLAN_ROUNDS *more* full ladders before being honestly rejected.
+
+Git is the exact record. Each commit of a record is one execution attempt, so we
+replay them:
+  * an attempt with ``ok=false``            -> total_attempts += 1
+  * ``fix_rounds`` dropping to 0 while the
+    tier does NOT rise (the reset_fix_loop
+    signature)                              -> replan_rounds += 1
+  * ``ok=true``                             -> both counters reset (success is clean)
+
+Idempotent: re-running recomputes from history rather than accumulating. Run from the
+repo root:  python scripts/backfill_execution_churn_counters.py [--apply]
+"""
+
+from __future__ import annotations
+
+import json
+import subprocess
+import sys
+from pathlib import Path
+
+STATE = Path("state/execution_status")
+
+
+def _history(path: Path) -> list[dict]:
+    shas = subprocess.run(
+        ["git", "log", "--format=%H", "--reverse", "--", str(path)],
+        capture_output=True, text=True, check=True,
+    ).stdout.split()
+    out = []
+    for sha in shas:
+        blob = subprocess.run(
+            ["git", "show", f"{sha}:{path}"], capture_output=True, text=True
+        ).stdout
+        try:
+            out.append(json.loads(blob))
+        except json.JSONDecodeError:
+            continue
+    return out
+
+
+def replay(revisions: list[dict]) -> tuple[int, int]:
+    """-> (total_attempts, replan_rounds) reconstructed from the record's history."""
+    attempts = replans = 0
+    prev_rounds = prev_tier = 0
+    for rec in revisions:
+        rounds = rec.get("fix_rounds") or 0
+        tier = rec.get("model_tier") or 0
+        if rec.get("ok") is True:
+            attempts = replans = 0            # a clean run wipes the churn
+        else:
+            if rounds > prev_rounds:
+                attempts += rounds - prev_rounds
+            elif rounds < prev_rounds and tier <= prev_tier:
+                # fix_rounds fell WITHOUT a tier bump => reset_fix_loop => a re-plan.
+                replans += 1
+                attempts += rounds           # the rounds accrued since the reset
+        prev_rounds, prev_tier = rounds, tier
+    return attempts, replans
+
+
+def main() -> int:
+    apply = "--apply" in sys.argv
+    changed = 0
+    for path in sorted(STATE.glob("*.json")):
+        revs = _history(path)
+        if not revs:
+            continue
+        attempts, replans = replay(revs)
+        rec = json.loads(path.read_text(encoding="utf-8"))
+        if rec.get("ok") is True:
+            attempts = replans = 0
+        if rec.get("total_attempts") == attempts and rec.get("replan_rounds") == replans:
+            continue
+        print(
+            f"{path.stem[:46]:48s} attempts={attempts:3d} replans={replans}"
+            f"  (fix_rounds={rec.get('fix_rounds')} tier={rec.get('model_tier')})"
+        )
+        changed += 1
+        if apply:
+            rec["total_attempts"] = attempts
+            rec["replan_rounds"] = replans
+            path.write_text(json.dumps(rec, indent=2) + "\n", encoding="utf-8")
+    print(f"\n{changed} record(s) {'updated' if apply else 'would change (dry run)'}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

src/llmxive/agents/lifecycle.py

@@ -105,9 +105,16 @@
     # deterministic report) instead of escalating to a human (autonomous
     # exhaustion handling). HUMAN_INPUT_NEEDED is retained only for unrelated
     # legacy markers — the execution path never routes there anymore.
+    # VALIDATOR_REJECTED is the terminal for a project whose analysis cannot be
+    # RUN here at all: it has exhausted every model tier AND every re-plan
+    # (MAX_REPLAN_ROUNDS full ladders). Re-planning it again would be an unbounded
+    # loop — which is exactly what it was, burning the worker matrix on a project
+    # that can never produce a real result. Rejecting is the honest outcome, and it
+    # is still autonomous (never HUMAN_INPUT_NEEDED).
     Stage.IN_PROGRESS: {
         Stage.RESEARCH_COMPLETE, Stage.IN_PROGRESS,
         Stage.PLANNED, Stage.HUMAN_INPUT_NEEDED,
+        Stage.VALIDATOR_REJECTED,
     },
     # research_complete is now a brief checkpoint where the
     # specialist reviewers run before research_review, so we allow

src/llmxive/execution/analysis_runner.py

@@ -66,6 +66,7 @@ class AnalysisRunResult:
     deadline_exceeded: bool = False
     reason: str = ""
     fabrication: list[str] = field(default_factory=list)         # deterministic fabricated-result findings
+    hollow: list[str] = field(default_factory=list)              # results that were never COMPUTED (null/NaN/empty)
 
 
 def extract_run_commands(quickstart_text: str) -> list[str]:
@@ -327,21 +328,48 @@ def run_analysis(
     # reaches research_complete and only the LLM panel catches it. A non-empty
     # finding hard-fails the gate → kickback to implementation for a REAL run.
     from llmxive.execution.fabrication_guard import find_fabrication
+    from llmxive.execution.hollow_guard import (
+        find_hollow_results,
+        find_no_durable_evidence,
+    )
 
     fabrication = find_fabrication(project_dir)
+    # DETERMINISTIC hollow-results gate: `bool(produced)` only asks "did a file
+    # appear?" — it never looks INSIDE. fabrication_guard catches numbers that were
+    # FAKED; this catches numbers that were never COMPUTED. PROJ-179 (metacognitive
+    # awareness, run on the IRIS FLOWER dataset) reached research_complete having
+    # written correlation=null, p=null, d_prime=NaN, robustness=[] and its own
+    # {"status": "PASS"}. Every headline number was missing and the gate said ok.
+    hollow = find_hollow_results(project_dir, produced)
+    # ...and a run whose every artifact is gitignored leaves nothing a reviewer can
+    # open or a paper can cite (PROJ-256 advanced on ONE data/processed/*.json).
+    # project_dir is <repo>/projects/<id>, so parents[1] is the repo whose .gitignore
+    # decides durability (the .gitignore is the SSoT — never re-encode its patterns).
+    undurable = find_no_durable_evidence(
+        project_dir, produced, repo_root=project_dir.parents[1]
+    )
     ok = (
         not deadline_exceeded
         and not cmd_failures
         and bool(produced)
         and not declared_missing
         and not fabrication
+        and not hollow
+        and not undurable
     )
     reason_parts: list[str] = []
     if fabrication:
         reason_parts.append(
             f"{len(fabrication)} fabricated/simulated-result signal(s) — results are "
             "not real measurements: " + "; ".join(fabrication[:3])
         )
+    if hollow:
+        reason_parts.append(
+            f"{len(hollow)} hollow-result signal(s) — the analysis ran but computed "
+            "nothing: " + "; ".join(hollow[:3])
+        )
+    if undurable:
+        reason_parts.append(undurable[0])
     if deadline_exceeded:
         reason_parts.append(f"overall deadline {overall_deadline_s:.0f}s exceeded")
     if cmd_failures:
@@ -372,6 +400,7 @@ def run_analysis(
         deadline_exceeded=deadline_exceeded,
         reason="; ".join(reason_parts) or "ok",
         fabrication=fabrication,
+        hollow=[*hollow, *undurable],
     )