Skip to content

feat(impact-certificate): certify a change's cross-boundary reach before it lands (spec-store arc, change 3) - #181

Merged
clay-good merged 9 commits into
mainfrom
feat/change-impact-certificate
Jun 22, 2026
Merged

clay-good merged 9 commits into
mainfrom
feat/change-impact-certificate

Conversation

@clay-good

@clay-good clay-good commented Jun 21, 2026

Copy link
Copy Markdown
Owner

Third and final change of the spec-store integration arc (openspec/changes/SPEC-STORE-INTEGRATION.md), stacked on #180 (change 2). Adds the change-impact certificate: for the current diff, OpenLore emits ONE deterministic, conclusion-shaped certificate of what the change touches — before it lands.

Stacked on #180. Review after #180 merges (GitHub will retarget this to main automatically). The change-3-only diff is the two commits on this branch.

What it does

change_impact_certificate (MCP tool) + openlore impact-certificate (CLI) compose, for a proposed change:

  • Blast radius / tests / drift — reused verbatim from blast_radius.
  • Newly-opened paths into each declared covering surface (a semantic/governance boundary you declare, not a directory glob) — the paths the change makes newly reachable, computed differentially (reachable after but not before). This is the cross-boundary signal file-ownership can't see.
  • Decay — the certificate is anchored to the touched symbols via the existing freshness lease, persisted under .openlore/impact-certificates/, and the spec-store health check re-fires a stale one as a certificate-stale finding. An expired certificate is never presented as silently still-true.

Advisory by default (never blocks); opt-in blocking only on a configured surface severity (e.g. impactCertificate.block: ["critical"]). No LLM — north star c6d1ad07 holds.

The differential, without the unbuilt incremental graph

The proposal specified deriving the post-change graph via add-watch-incremental-dependency-graph, which is still a DRAFT/unbuilt. A new call edge can only originate from a changed file, so this re-parses only the changed files at base vs working tree (the same bounded primitive structural_diff uses), resolves the added/removed callee names to canonical ids by unique-name match, and adjusts the canonical adjacency both ways:

post = canonical + addedEdges − removedEdges
pre  = canonical − addedEdges + removedEdges

A node that reaches a surface in post but not pre is newly able to — and the opening is attributed to the exact added edge, with its shortest path named. No full rebuild, no dependency on the unbuilt graph. Ambiguous added callees are reported (unresolved-added-call), never guessed. Recorded as decision 187224b0.

Verification

  • Unit: impact-certificate.test.ts — 17 tests: surface resolution (file+symbol, unresolved→finding, ambiguous→no-guess), both differential scenarios (opens a 2-hop path / touches only existing callers → none) + direct + already-reachable + empty, the block gate, conclusion-shape + contract classification, and decay fresh→stale against a real on-disk edge store + the spec-store health-check re-fire.
  • e2e dogfood on this repo (6,286 functions) — caught a controlled critical newly-opened path dogfoodNewlyOpensSurface → validateSpecStoreConfig, blocked the hook at exit 1 under block: ["critical"] and stayed advisory (exit 0) without it, persisted 31 lease anchors, and turned a certificate stale on an anchored-symbol edit. See openspec/changes/DOGFOOD-change-impact-certificate.md.
  • Full CI-equivalent suite: vitest run src examples (integration excluded) — 4,376 passed / 2 skipped.

Surface registration

New tool joins the opt-in federation preset only (out of minimal/navigation/memory). Tool count 62→63; tools/list budget, doc size/count figures, presets/contract/tool-driver guards all updated consciously.

Adversarial hardening (round 2)

Two independent reviewers + real-input e2e probes found and fixed two correctness bugs in the changed-file plumbing (both from computeEdgeDelta diverging from the sibling structural_diff) plus one no-throw gap:

  • Rename false positive (HIGH): the diff dropped oldPath, so a renamed file's pre-existing calls all looked added — a pure git mv falsely reported a newly-opened critical path and would have wrongly blocked the commit under block: ["critical"]. Fixed: read old content from oldPath ?? path.
  • Untracked false negative (MEDIUM): git diff excludes untracked files, so a brand-new file opening a critical surface was certified as "no new reach." Fixed: fold in git ls-files --others --exclude-standard (as structural_diff does).
  • No-throw gap (MEDIUM): the decay re-check could throw out of the contractually no-throw handleSpecStoreStatus. Fixed: catch at the source + a boundary at the call site; an unverifiable certificate is conservatively stale.

All three reproduced with real inputs, fixed, and pinned by 4 new regression tests against a real temp git repo + the real CallGraphBuilder (17 → 21 tests). See the Round 2 section of DOGFOOD-change-impact-certificate.md.

Adversarial hardening (round 3 — second pass)

A second adversarial round (two fresh reviewers + new real-input probes) found two more correctness bugs, a file-hygiene defect, and doc/test gaps. All fixed and regression-tested.

  • Homonym phantom opening (HIGH): resolving an added call's callee by name against the canonical graph mis-bound a LOCAL helper sharing a covering-surface symbol's name to the canonical surface — a phantom newly-opened path that falsely tripped the critical block-gate (reproduced e2e). Fixed: key changed-file calls by their snapshot-internal resolved id and only name-resolve callees external to the snapshot, honoring the local binding.
  • Same-diff surface member missed (MEDIUM): a surface symbol added in the same diff was unresolvable (canonical-only), silently downgrading a critical opening to a warn. Fixed: resolve surfaces over canonical ∪ post-change nodes.
  • Three stray NUL bytes (accidental key separators) made the handler read as binary and hid it from grep; replaced with the equivalent escape.
  • Docs: the tool/command previously appeared only in CLAUDE.md; now fully documented in docs/mcp-tools.md (table + prose + parameters + finding codes), docs/cli-reference.md, README.md, and docs/federation.md.
  • Tests: added a CLI test file (hook install/uninstall + advisory/blocking exit codes), a dispatchTool MCP-path reachability test, and homonym + same-diff regression tests against a real temp git repo. Full suite: 4,399 passed / 2 skipped.

Adversarial hardening (round 4 - third pass)

A third adversarial pass (scale/determinism/correctness reviewer + new real-input probes) found two more HIGH correctness bugs and several integrity gaps. All fixed + regression-tested.

  • Base-ref divergence (HIGH): getChangedFiles diffs against the merge-base (three-dot) but the differential read old content from the base-ref tip - so when the base branch advanced, the certificate's two halves diffed against different commits (phantom/missed openings). Fixed: read old content from git merge-base(base, HEAD); pinned in a real temp git repo.
  • Wrong-typed surface severity (HIGH): an out-of-enum severity made highestSurfaceSeverity come out null (NaN rank index), breaking the block-signal contract. Fixed: coerce to warn in surfacesFromConfig (reproduced e2e: was null, now warn).
  • Duplicate/empty surface names dropped; a member with both symbol+file resolves both; total-order path sort (deterministic top-N); per-surface truncation now reports the true count + a caveat (no silent truncation); large-diff caveat; and a new/untracked file now gets a file-level lease anchor so the certificate actually decays for it.
  • Docs: docs/configuration.md now documents the impactCertificate config (surfaces, members, severity, block) - the last doc surface that lacked it.
  • Suite: 4,405 passed / 2 skipped (merge-base, severity coercion, dedup, both-member, file-anchor-decay regressions added).

Round 5 - doc accuracy + MCP-server integration (4th pass)

A fourth pass (spec/doc-accuracy + integration-coverage reviewer + a real large-diff perf probe) found no new runtime bugs - the differential, decay, config hardening, and finding codes were all confirmed accurate - and fixed documentation + integration-coverage gaps:

  • Doc accuracy: five normative spec/proposal bodies still asserted the un-shipped "incremental dependency graph" mechanism (contradicting the shipped differential edge-delta and their own notes). Reworded all five (canonical mcp-handlers requirement, the change-delta spec, proposal items, tasks.md) to be mechanism-neutral; extended the lease note to record the new-file file-level anchor.
  • Integration coverage: the spec-12 conformance test ran the live MCP stdio server but its ListTools check was one-directional. Made it bidirectional (every defined tool is advertised) + a positive assert that change_impact_certificate is advertised — 7/7 conformance tests pass, confirming the tool is reachable end-to-end through the real MCP server (not just unit dispatch). Protects all tools.
  • Perf probe (real input): a 300-file diff completes in ~8.8s, 300/300 openings detected, no hang — validating the round-4 >200-file caveat (capping is correctly avoided; it would miss openings).
  • Finding codes / CLI flags confirmed accurate; added the --uninstall-hook doc example.

🤖 Generated with Claude Code

sim and others added 6 commits June 21, 2026 17:14
…ore it lands (spec-store arc, change 3)

Adds change_impact_certificate (MCP tool + `openlore impact-certificate` CLI):
for the current diff, emit ONE deterministic, conclusion-shaped certificate —
blast radius (reused from blast_radius), the paths the change NEWLY OPENS into
each declared covering surface, drifted specs, and tests to run.

Newly-opened-path detection is differential and needs no full rebuild and no
(unbuilt) incremental graph: a new call edge can only originate from a changed
file, so re-parse only the changed files at base vs working tree, take each
caller's added/removed callee names, resolve them to canonical ids by
unique-name match, and adjust the canonical adjacency both ways
(post = canonical + added - removed, pre = canonical - added + removed). A node
that reaches a surface in post but not pre is newly able to — the path opened.

The certificate decays via the existing code-anchored freshness lease: it is
anchored to the touched symbols, persisted under .openlore/impact-certificates/,
and the spec-store health check re-fires a stale one as a finding. Advisory by
default; opt-in blocking only on a configured surface severity. No LLM.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ogfood report; clean path naming

- Merge the change-impact-certificate ADDED requirements into the canonical
  mcp-handlers + cli specs (CoveringSurfaceDeclaration, NewlyOpenedPathDetection,
  ChangeImpactCertificate, ImpactCertificateDecaysWithLease, ImpactCertificateCommand),
  each annotated with the scoped differential-edge-delta deviation.
- Flip proposal + tasks to IMPLEMENTED; mark the spec-store arc COMPLETE (all 3 shipped).
- Add CLAUDE.md tool-table row + DOGFOOD-change-impact-certificate.md (real e2e on this repo).
- Polish: a newly-added caller absent from the index now renders its bare symbol
  name in the opening path, not its path-based node id.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…iles, no-throw decay

Two correctness bugs found by adversarial review + real-input e2e, both from
computeEdgeDelta diverging from the sibling structural_diff that gets these right:

- RENAME false positive (HIGH): the diff dropped getChangedFiles' oldPath, so a
  renamed file's old content was read by its NEW path (git show fails) → every
  pre-existing call looked "added" → a pure rename falsely reported a newly-opened
  path (and would have wrongly BLOCKED the commit under block:["critical"]).
  Fix: ChangedFileEntry carries oldPath/status; old content reads from oldPath ?? path.

- UNTRACKED false negative (MEDIUM): git diff excludes untracked files, so a
  brand-new (un-git-added) file opening a path into a surface was silently missed —
  the certificate certified "no new reach" while a real critical opening existed.
  Fix: fold in `git ls-files --others --exclude-standard` (as structural_diff does).

- NO-THROW gap (MEDIUM): recheckCertificate / recheckPersistedCertificates could
  throw (corrupt target anchor graph, wrong-typed lease) out of the contractually
  no-throw handleSpecStoreStatus. Fix: catch at the source + a try/catch boundary
  at the spec-store call site; an unverifiable certificate is conservatively stale.

Regression-tested against a real temp git repo + the real CallGraphBuilder snapshot
(17 → 21 tests: rename opens nothing, untracked detected, in-place edit detected,
corrupt cert never throws). Full CI-equivalent suite: 4,380 passed / 2 skipped.
Dogfood Round 2 + the diff-completeness guarantee recorded in the specs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rface miss; full docs + CLI tests

Second adversarial round (two reviewers + real-input probes) on PR #181.

Correctness:
- HOMONYM PHANTOM OPENING (HIGH): resolving an added call's callee by NAME against
  the canonical graph mis-bound a LOCAL helper sharing a surface symbol's name to
  the canonical surface - a phantom newly-opened path that falsely tripped the
  critical block-gate (reproduced e2e). Fix: key changed-file calls by their
  snapshot-internal resolved id and only name-resolve callees external to the
  snapshot, honoring the snapshot's own local binding. (Decision 97c22605.)
- SAME-DIFF SURFACE MEMBER MISSED (MEDIUM): a surface symbol added in the same diff
  was unresolvable (canonical-only), silently downgrading a critical opening to a
  warn. Fix: resolve surfaces over canonical + post-change snapshot nodes; compute
  the edge delta before surface resolution.
- Three stray NUL bytes (accidental key separators) made the file read as binary and
  hid it from grep - replaced with the unicode escape (identical runtime, valid UTF-8).

Docs (previously only in CLAUDE.md): change_impact_certificate / openlore
impact-certificate now documented in docs/mcp-tools.md (table + prose + parameters +
finding codes), docs/cli-reference.md (table + command section), README.md, and
docs/federation.md.

Tests: new src/cli/commands/impact-certificate.test.ts (hook install/uninstall +
advisory/blocking exit codes), a dispatchTool MCP-path reachability test, and
regression tests for the homonym + same-diff-surface bugs against a real temp git
repo. Full CI-equivalent suite: 4,399 passed / 2 skipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ng (3rd adversarial pass)

Third adversarial round on PR #181 found two HIGH correctness bugs + integrity gaps.

- BASE-REF DIVERGENCE (HIGH): getChangedFiles diffs against the MERGE-BASE
  (three-dot base...HEAD) but the differential read old content from the base-ref
  TIP. When the base branch advanced, the certificate's two halves diffed against
  different commits -> phantom or missed openings. Fix: read old content from
  git merge-base(base, HEAD). Reproduced + pinned in a real temp git repo.
- WRONG-TYPED SEVERITY (HIGH): an out-of-enum surface severity made
  highestSurfaceSeverity come out null (NaN SEVERITY_RANK index), breaking the
  block-signal contract. Fix: coerce to 'warn' in surfacesFromConfig (+ ?? 0 guard).
  Reproduced e2e (was null, now 'warn').
- Duplicate surface names (collided in the findings map, dropped a severity) and
  empty/whitespace names are now dropped; a member with both symbol+file resolves
  both; the newly-opened-paths sort is a total order (deterministic top-N);
  per-surface path truncation reports the TRUE count + a caveat (no silent
  truncation); a large-diff caveat is emitted; and a new/untracked file (no indexed
  symbol) gets a FILE-level lease anchor so the certificate actually decays for it.
- Fixed two stray SOH bytes (raw \x01 sort separators) that re-flagged the file as
  binary; replaced with the escape.

Docs: docs/configuration.md now documents the impactCertificate config (surfaces,
members, severity, block) - the last doc surface that lacked it.
Tests: merge-base baseline (real git repo), severity/duplicate/empty-name coercion,
both-member resolution, file-level-anchor decay. Suite: 4,405 passed / 2 skipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-server integration gap (4th pass)

Fourth adversarial pass (doc-accuracy + integration-coverage reviewer + a real
large-diff perf probe). No new runtime bugs; the differential/decay/config/finding
codes were all confirmed accurate. Fixed doc inaccuracies + one integration gap.

- DOC ACCURACY: three rounds of fixes settled the implementation on a differential
  edge-delta over changed files, but five normative spec/proposal BODIES still
  asserted the un-shipped "incremental dependency graph" as the mechanism
  (contradicting the code and their own notes). Reworded all five to be
  mechanism-neutral / name the differential: canonical mcp-handlers requirement,
  the change-delta spec (added a note where none existed), proposal items 2 and
  "Application to OpenLore", and tasks.md item 2. Extended the lease note to record
  that new/untracked files get a file-level anchor.

- INTEGRATION COVERAGE: the spec-12 conformance test ran the live MCP stdio server
  but its ListTools check was one-directional (advertised subset of known). Made it
  BIDIRECTIONAL (every defined tool is advertised) + a positive assertion that
  change_impact_certificate is advertised. Verified: 7/7 conformance tests pass, so
  the tool is now confirmed reachable end-to-end through the real MCP server, not
  just the unit-level dispatchTool path. The bidirectional check protects all tools.

- Perf probe (real input): a 300-file diff completes in ~8.8s (300 git-show + 2
  tree-sitter builds), 300/300 openings detected, no hang - validating the round-4
  >200-file caveat (capping is correctly avoided; it would miss openings).

- docs/cli-reference.md: added the --uninstall-hook example for symmetry.

Suite: 4,405 passed / 2 skipped; conformance integration: 7 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@clay-good
clay-good deleted the branch main June 22, 2026 00:02
@clay-good clay-good closed this Jun 22, 2026
…rtificate

# Conflicts:
#	README.md
#	docs/cli-reference.md
#	docs/governance-dogfooding.md
#	docs/mcp-tools.md
#	openspec/specs/cli/spec.md
#	src/cli/commands/mcp.ts
@clay-good clay-good reopened this Jun 22, 2026
@clay-good
clay-good changed the base branch from feat/working-set-context-briefing to main June 22, 2026 00:05
@clay-good
clay-good merged commit 49ec719 into main Jun 22, 2026
4 checks passed
@clay-good
clay-good deleted the feat/change-impact-certificate branch June 22, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant