Skip to content

fix(556): gate schema-echo dicts at the persistence boundary#803

Merged
Muizzkolapo merged 3 commits into
mainfrom
fix/556-schema-echo-persistence-gate
Jul 18, 2026
Merged

fix(556): gate schema-echo dicts at the persistence boundary#803
Muizzkolapo merged 3 commits into
mainfrom
fix/556-schema-echo-persistence-gate

Conversation

@Muizzkolapo

Copy link
Copy Markdown
Owner

Summary

StorageBackend.write_target now refuses to persist a record whose action namespace is the compiled JSON Schema (the Ollama format shape: {"title","type":"object","properties",...}) instead of LLM output. A resume / carry-forward record reaches write_target without an LLM call, so the in-flight _reject_schema_echo_items guard never runs and the schema dict was written verbatim into target_data.

The new _gate_schema_echo_deltas runs after delta extraction and before _write_target_raw: a namespace matching is_schema_echo is replaced with the make_schema_echo_error sentinel and the record gets a FAILED/PARSE_ERROR disposition, so the failure surfaces on the next read exactly like the live-LLM parse-error path. It lives on the abstract base, so every backend inherits the invariant. Reuses is_schema_echo / make_schema_echo_error / PARSE_ERROR as-is — no new constants, no collector change.

Verification

  • RED: 5 of 7 tests fail on main (echo persisted verbatim, no disposition), including a varied-action-name test so a hardcoded-name gate can't pass.
  • GREEN: full suite 8126 passed; ruff + mypy clean.
  • Review (HIGH tier, 3 blind lenses):
    • Correctness: YES — gate at the right chokepoint (covers force_full + delta branches), guards mirror _extract_delta, replacement reaches _write_target_raw, NotImplementedError caught.
    • Blast radius: YES — sqlite RLock is re-entrant (calling set_disposition from inside write_target is deadlock-safe); every write_target caller checked for false-positives (none carry title+type:object+properties); no SUCCESS-write race.
    • Tests/anti-gaming: flagged that all tests used one action name → fixed by adding test_gate_fires_for_any_action_name (recorded in RED). Real SQLite round-trip, non-tautological assertions, collector untouched.
  • Smoke: skipped — scan-project canary divergence is pre-existing (baseline.sh confirms it fails on main too; stale project action registry, unrelated to storage). The gate is a no-op on clean data; the real write_target_write_target_raw_read_target_raw round trip + disposition writes are covered by the 7 unit tests.

Deviations from the spec (verify-first)

  • set_disposition clears priors. The spec assumed a prior SUCCESS and the gate's FAILED would coexist (UNIQUE(action,record,disposition)). Current set_disposition (sqlite_backend.py) DELETEs all prior dispositions for (action, record) first, so the gate's FAILED replaces a stale SUCCESS — the record ends unambiguously failed (better). Test asserts the real behavior.
  • Test path is tests/unit/storage/ (spec said tests/storage/).
  • Real-DB before-count is now 0 (spec cited 11 rows on 2026-06-17). Those rows were cleaned since; the gate is a forward-looking invariant — RED proves write_target still persists a schema-echo namespace verbatim on main, i.e. the hole is open.

Out of scope (follow-up)

The upstream planter that stamps compile_unified_schema output as record content on the resume/checkpoint path — suspect sites processing/strategies/online_llm.py:_checkpoint_record and llm/batch/services/processing_recovery.py. This gate is the safety net regardless of which path is wrong.

Schema-echo dicts reach target_data verbatim via the carry-forward/resume
path (no LLM call, so the in-flight _reject_schema_echo_items guard never
runs). Asserts write_target replaces the namespace with the parse-error
sentinel and records FAILED/PARSE_ERROR — across a varied action name so a
hardcoded-name gate cannot pass.
A resume/carry-forward record can reach write_target with its action namespace
set to the compiled JSON Schema (Ollama format shape) instead of LLM output —
the LLM was never invoked, so the in-flight _reject_schema_echo_items guard
never ran and the schema dict was persisted verbatim to target_data.

StorageBackend.write_target now runs _gate_schema_echo_deltas after delta
extraction and before _write_target_raw: a namespace matching is_schema_echo is
replaced with the make_schema_echo_error sentinel and the record gets a
FAILED/PARSE_ERROR disposition. Backend-agnostic (lives on the abstract base).

Deviations from the spec: set_disposition now clears prior dispositions for
(action, record), so the gate's FAILED replaces a stale SUCCESS rather than
coexisting; test path is tests/unit/storage/ (spec said tests/storage/); the
real-DB before-count is now 0 (the 11 rows from 2026-06-17 were cleaned) — the
gate remains a forward-looking invariant (RED proves the hole was open).
@Muizzkolapo

Copy link
Copy Markdown
Owner Author

Real-project verification (added)

Ran the gate against a copy of the actual qanalabs code_centered_quiz.db (real SQLiteBackend, real write_target chokepoint — not an in-memory fixture). Wrote a synthetic schema-echo record for explain_distractor_1:

[explain_distractor_1] Schema-echo in delta for source_guid=sg-demo (keys: [...]) — replacing with parse-error sentinel
persisted namespace keys: ['_parse_error', 'raw_response']
  -> schema echo gone?  True
  -> parse-error sentinel present?  True
disposition rows: [('failed', 'parse_error')]

Confirms the gate replaces the namespace and records FAILED/PARSE_ERROR end-to-end against the real project DB schema. Run on a copy; the project DB was not mutated.

Note: the spec's original before-check (11 rows on 2026-06-17) now returns 0 on the live DB — those rows were cleaned since. The gate is a forward-looking invariant (RED proves write_target still persists a schema-echo namespace verbatim on main).

Post-review cleanup (no behavior change):
- Collect the FAILED/PARSE_ERROR rows and write them via set_dispositions_batch
  (one transaction) instead of a per-record set_disposition call. Same
  DELETE-then-INSERT semantics, so a stale prior SUCCESS is still cleared.
- Move the record.reasons / utils.schema_echo imports to the module top-level;
  neither imports storage, so there is no cycle to defer around, and it matches
  the file's import convention.
@Muizzkolapo
Muizzkolapo merged commit cb0edfb into main Jul 18, 2026
5 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 18, 2026
@Muizzkolapo
Muizzkolapo deleted the fix/556-schema-echo-persistence-gate branch July 18, 2026 06:38
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant