Skip to content

[MC-456-B] docs: persistence model — runtime read path, block shapes, served-but-stale troubleshooting#647

Merged
TrueNorth49 merged 3 commits into
mainfrom
docs/mc-456-b-persistence-model
Jun 2, 2026
Merged

[MC-456-B] docs: persistence model — runtime read path, block shapes, served-but-stale troubleshooting#647
TrueNorth49 merged 3 commits into
mainfrom
docs/mc-456-b-persistence-model

Conversation

@TrueNorth49

Copy link
Copy Markdown
Collaborator

What

Expands docs/data-persistence-model.md. The doc explained the concept-key invariant and migrations well, but was disk-centric — it documented neither the runtime request path nor the value shapes of each block. That is precisely the axis where the "the file is correct but the UI shows nothing" class of problem lives, so the doc could not answer it.

All additions are language- and data-agnostic: generic conceptKey / speakerId / GROUP placeholders, no dataset- or corpus-specific lexemes. Mechanism is described generically; the only concrete names are structural (file names, endpoints, env var).

Sections added

  1. The parse-enrichments.json Block Map (shapes + precedence) — every top-level (automatic) and manual_overrides (human) block with its value shape and key namespace; the manual-wins precedence rule; and the explicit consequence that a reader inspecting the top-level cognate_sets reports 0 for a hand-authored workspace even though manual_overrides is full.
  2. How the Running App Reads This Data (Request Path) — configurable backend port (PARSE_PORT, default 8766), per-request re-read with in-memory SAFE-key promotion, the GET /api/enrichments{ "enrichments": {…} } wrapper, GET /api/compare/bundles carrying grouping/forms but not cognate letters, how the UI joins the two by concept key, and the in-memory-snapshot / stale-client caveat for any out-of-band rewrite (migration, external authoring/merge tool, hand edit, restore).
  3. Troubleshooting: the file is correct but the UI or a report disagrees — ordered checklist (stale client → layer/precedence → key → speaker id → surface/workspace → endpoint shape).

Also: header Last updated note bumped; a cross-reference added from the Core Files table row to the new Block Map.

Scope

Documentation only — no code or behavior change.

Verification

  • Markdown structure verified (section headers, table integrity).
  • Block shapes taken from the authoritative type definitions (src/lib/decisionPersistence.ts: CognateSetMap, SpeakerFlagMap, BorrowingFlagMap, CognateDecisionMap, CanonicalLexemesByBundle, concept_merges) and the backend default-enrichments skeleton; the endpoint shapes confirmed against the live /api/enrichments and /api/compare/bundles responses.

🤖 Generated with Claude Code

… served-but-stale troubleshooting

The persistence/keying doc explained keys and migrations well but was
disk-centric: it did not document the request path or the value shapes,
which is exactly where "the file is correct but the UI looks empty"
lives. Adds three sections (all language/data-agnostic — generic
conceptKey/speakerId/GROUP placeholders, no dataset specifics):

- Block Map: every top-level (automatic) and manual_overrides (human)
  block with its value shape, the manual-wins precedence rule, and the
  consequence that a tool reading the top-level cognate_sets reports 0
  for a hand-authored workspace even when manual_overrides is full.
- Request Path: backend port (PARSE_PORT), per-request re-read, the
  /api/enrichments { "enrichments": {...} } wrapper, /api/compare/bundles
  carrying grouping/forms but NOT cognate letters, how the UI joins them
  by concept key, and the in-memory-snapshot / stale-client caveat for
  any out-of-band rewrite.
- Troubleshooting: ordered checklist for "data on disk but not shown"
  (stale client → layer/precedence → key → speaker id → surface/workspace
  → endpoint shape).

Docs only; no code or behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@TrueNorth49 TrueNorth49 added docs Documentation changes MC-456 Mission Control MC-456 labels Jun 2, 2026
discarded_forms is { conceptKey: { speakerId: <reason> } } — a per-speaker
free-text exclusion reason, not an opaque value. Verified against the writer
and live data.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@TrueNorth49

Copy link
Copy Markdown
Collaborator Author

Review — PR #647 (self-review, MC-456-B)

Overview. Docs-only (+~70/−2, one file: docs/data-persistence-model.md). Adds a Block Map (every top-level/automatic and manual_overrides/human block with its value shape + the manual-wins precedence), a Request-Path section (port, per-request re-read, /api/enrichments wrapper, /api/compare/bundles contents, UI join, stale-client caveat), and a Troubleshooting checklist. Closes the gap that made the original doc unable to answer "the file is correct but the UI is empty."

Correctness (verified against the code/runtime, not assumed):

  • Block shapes ✓ against src/lib/decisionPersistence.ts types (CognateSetMap, SpeakerFlagMap, BorrowingFlagMap, CognateDecisionMap, CanonicalLexemesByBundle) + the backend default-enrichments skeleton.
  • cognate_decisions legacy top-level fallback ✓ (getCanonicalManualOverrides).
  • Manual-wins precedence ✓ (overrideSets[key] ?? autoSets[key], speakerForm.ts).
  • UI key resolution variants[selectedIdx].conceptKey ?? concept.key ✓ (speakerForm.ts).
  • GET /api/enrichments { "enrichments": {…} } wrapper and GET /api/compare/bundles shape ✓ confirmed against live responses.
  • Default port 8766 / PARSE_PORT ✓ (server.py, parse-run).
  • canonical_lexemes bundle-keyed (not concept-keyed) ✓ matches the migration's CONCEPT_KEYED_BLOCKS exclusion.

Fix applied during review (per AGENTS.md PR review rule 3): discarded_forms was under-specified as { conceptKey: … }; verified the real shape is { conceptKey: { speakerId: <reason> } } (per-speaker free-text exclusion reason) against the writer + live data and corrected it (commit 714647d).

AGENTS.md review rules:

  1. Language/data-agnostic ✓ — generic conceptKey/speakerId/GROUP/bundleKey placeholders throughout; no language, speaker, or corpus appears. Concrete names are structural only (file names, endpoints, env var).
  2. Human-readable ✓ — short, leads with the mechanism; tables for shapes; ordered checklist for triage.
  3. Docs assessment — this PR is the docs change; no further docs warranted.

Risk: none to runtime — documentation only.

Verdict: approve. Accurate, generic, and self-contained. Leaving merge to the maintainer.

…cs/skills) + cross-link

.claude/ is gitignored, so the skill ships as a tracked doc under
docs/skills/. Generic, data/language-agnostic runbook paired with
docs/data-persistence-model.md: correct concept key, manual-vs-automatic
precedence, legacy-key migration, and end-to-end verification for
updating cognate decision data in the main PARSE app. Linked from the
persistence doc's Related Reading.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@TrueNorth49 TrueNorth49 merged commit 071198d into main Jun 2, 2026
4 checks passed
@TrueNorth49 TrueNorth49 deleted the docs/mc-456-b-persistence-model branch June 2, 2026 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation changes MC-456 Mission Control MC-456

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant