daemon: out-of-band checkpoint anchor for tail-truncation resistance#871
Merged
Conversation
Spike (#600) closing the ADR-0008 §2 tail-truncation gap with an additive, out-of-band signed checkpoint emitted to a pluggable multi-sink anchor. Receipts stay a linear VC chain — nothing here touches the receipt schema, hash chain, @context, or issuer DID. - checkpoint package: {chain_id, sequence, receipt_hash, timestamp} signed via the existing RFC 8785 JCS path; Sign/Verify against the daemon key. - Emitter: fan-out to a sink LIST, per-chain cadence (default every receipt) + graceful-shutdown flush; sink failures are logged + metered (fail-visible, never blocking receipt emission — the opposite of the PY-P9 silent drop). - Sinks behind the existing anchor.Sink interface: git (commit chain = the only Merkle structure; agent UID cannot write), file (FileLog), syslog (unix). OpenSink/OpenSinks dispatch file:/git:/syslog: specs. - Daemon wiring: --checkpoint-anchor (fan-out list) + --checkpoint-cadence, with env + TOML parity; emit after each commit, flush on shutdown. Empty config keeps the commit path byte-identical to before. - verify --against-anchor: chain intact + checkpoint signatures + strictly increasing log + head-match; a checkpoint ahead of the store HEAD is reported FAIL (truncation). Default off — verify is unchanged without the flag. - CI gate (ADR-0024 catalogue #11): emit chain, anchor it, truncate the store tail, assert verify --against-anchor goes red with a truncation reason while plain verify still reports VALID. - Freeze: comment at the emit site + a one-line note in ADR-0008 that receipts MUST NOT carry an anchor reference; anchoring is out-of-band by design.
…reader/PEM parse Review follow-ups on the checkpoint-anchor spike. Correctness: - Emitter now tracks the highest sequence already anchored per chain and gates both Observe and Flush on it. The shutdown Flush re-anchored the last head whenever the interrupted-chain terminator was skipped (tight/expired deadline), writing a second checkpoint at the same sequence — which `verify --against-anchor` then rejected as a non-strictly-increasing log, failing a healthy, untruncated chain. Flush of an already-anchored head is now a no-op. - GitLog applies its local config (user identity + commit.gpgsign=false) on every open, not only on fresh init, so an operator-provided repo can commit on a host that enforces signed commits, and an invalid .git fails loudly at open instead of silently on every write. Cleanup: - Move the anchor-reading + checkpoint-verify loop into checkpoint.ReadVerifiedCheckpoints; verify --against-anchor and the daemon end-to-end test now share it (removes a duplicated reader + bufio sizing). - Export checkpoint.PublicKeyFromPEM and use it from the verify CLI, collapsing the two daemon-side Ed25519 PEM parsers into one. Tests: emitter no-double-emit regression; abrupt-shutdown (1ns deadline) e2e regression asserting no duplicate checkpoint; pre-existing-repo git config test.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds an out-of-band, signed checkpoint anchoring mechanism to the daemon to detect receipt-store tail truncation (ADR-0008 gap), without changing the receipt schema or hash chain. It introduces a checkpoint emitter with pluggable multi-sink anchors (file/git/syslog) and extends obsigna receipt verify with an opt-in --against-anchor check plus CI gates/tests.
Changes:
- Introduces
daemon/internal/checkpoint(checkpoint format, signing/verification, emitter, anchor log reader) and wires it into the pipeline + daemon shutdown flush. - Adds anchor sink dispatch and new sink backends (git repo commits, file log reuse, syslog), plus config/flag/env/TOML plumbing.
- Adds
verify --against-anchorand a truncation gate test, plus docs/ADR updates describing the anchoring freeze and verification contract gate.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/adr/0024-project-verification-contract.md | Adds Gate #11 for tail-truncation detection via checkpoint anchoring. |
| docs/adr/0008-response-hashing-and-chain-completeness.md | Adds an “anchoring freeze” note: anchoring remains out-of-band and never enters receipts. |
| daemon/tests_framework.go | Adds DaemonFixture.Stop to allow tests to wait for graceful shutdown/flush completion. |
| daemon/tests_checkpoint_truncation_test.go | Adds deterministic truncation gate test that proves --against-anchor detects tail truncation. |
| daemon/tests_checkpoint_anchor_test.go | Adds integration tests covering end-to-end anchoring + shutdown flush and duplicate-avoidance regression. |
| daemon/README.md | Documents checkpoint anchoring, sink specs, cadence, failure behavior, and verify --against-anchor. |
| daemon/internal/verifycli/verify.go | Adds --against-anchor flag and invokes anchor verification after chain validation. |
| daemon/internal/verifycli/anchor.go | Implements the anchor verification logic (head match + truncation detection). |
| daemon/internal/verifycli/anchor_test.go | Unit tests for anchor verification outcomes (pass, truncation, tamper, missing, forged, ordering). |
| daemon/internal/pipeline/build.go | Adds pipeline Checkpointer hook and shutdown FlushCheckpoints. |
| daemon/internal/checkpoint/reader.go | Adds shared anchor-log reader that verifies checkpoint signatures while parsing NDJSON records. |
| daemon/internal/checkpoint/emitter.go | Adds per-chain cadence-based checkpoint emitter with fan-out sinks and failure metering/logging. |
| daemon/internal/checkpoint/emitter_test.go | Unit tests for fan-out, cadence+flush, fail-visible behavior, duplicate prevention, and close. |
| daemon/internal/checkpoint/checkpoint.go | Defines checkpoint payload + signature wrapper and Ed25519 sign/verify helpers. |
| daemon/internal/checkpoint/checkpoint_test.go | Crypto-level tests for sign/verify round-trip and malformed/wrong-key cases. |
| daemon/internal/anchor/syslog_unix.go | Adds syslog sink implementation (unix) using shared record envelope format. |
| daemon/internal/anchor/syslog_other.go | Adds non-unix stub returning a clear “syslog unsupported” error. |
| daemon/internal/anchor/open.go | Adds OpenSink / OpenSinks dispatcher for file/git/syslog sink specs. |
| daemon/internal/anchor/git.go | Adds git-backed sink that appends NDJSON and commits each write for tamper-evident history. |
| daemon/internal/anchor/git_test.go | Tests git sink write/commit behavior, config-on-open, and input validation. |
| daemon/internal/anchor/anchor.go | Adds checkpoint event type and refactors shared record-line marshaling for all sinks. |
| daemon/daemon.go | Wires checkpointing into daemon run lifecycle and flushes checkpoints on shutdown. |
| daemon/configfile.go | Extends TOML config schema with checkpoint anchor list + cadence. |
| daemon/cmd/obsigna-daemon/main.go | Adds flags/env/TOML parsing for checkpoint anchors/cadence and prints resolved config. |
| daemon/cmd/obsigna-daemon/config_test.go | Adds config-layer tests for checkpoint anchor parsing and cadence validation. |
- verify --against-anchor: drop the pre-sort and validate strict monotonicity in anchor FILE ORDER. The emitter only ever writes a head past the highest it already anchored, so file order is strictly increasing by construction and an out-of-order/duplicate record is itself the tamper signal — sorting first laundered "seq 2 then seq 1" into a passing "1,2". The last record in file order is therefore the genuine latest head. Adds an out-of-order regression test. - checkpoint.Verify: correct the docstring to distinguish (false, err) — the artifact could not be evaluated — from (false, nil) — parsed but the signature does not match. Behaviour unchanged; only the contract is now accurate. - syslog sink: strip recordLine's trailing newline before writing, so one checkpoint is one syslog message rather than risking a split/multi-line entry.
ojongerius
pushed a commit
that referenced
this pull request
Jun 16, 2026
- verify --against-anchor: drop the pre-sort and validate strict monotonicity in anchor FILE ORDER. The emitter only ever writes a head past the highest it already anchored, so file order is strictly increasing by construction and an out-of-order/duplicate record is itself the tamper signal — sorting first laundered "seq 2 then seq 1" into a passing "1,2". The last record in file order is therefore the genuine latest head. Adds an out-of-order regression test. - checkpoint.Verify: correct the docstring to distinguish (false, err) — the artifact could not be evaluated — from (false, nil) — parsed but the signature does not match. Behaviour unchanged; only the contract is now accurate. - syslog sink: strip recordLine's trailing newline before writing, so one checkpoint is one syslog message rather than risking a split/multi-line entry.
290cadd to
e02a75a
Compare
This was referenced Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spike (#600) closing the ADR-0008 §2 tail-truncation gap with an additive,
out-of-band signed checkpoint emitted to a pluggable multi-sink anchor.
Receipts stay a linear VC chain — nothing here touches the receipt schema,
hash chain, @context, or issuer DID.
the existing RFC 8785 JCS path; Sign/Verify against the daemon key.
graceful-shutdown flush; sink failures are logged + metered (fail-visible,
never blocking receipt emission — the opposite of the PY-P9 silent drop).
Merkle structure; agent UID cannot write), file (FileLog), syslog (unix).
OpenSink/OpenSinks dispatch file:/git:/syslog: specs.
with env + TOML parity; emit after each commit, flush on shutdown. Empty
config keeps the commit path byte-identical to before.
increasing log + head-match; a checkpoint ahead of the store HEAD is reported
FAIL (truncation). Default off — verify is unchanged without the flag.
tail, assert verify --against-anchor goes red with a truncation reason while
plain verify still reports VALID.
MUST NOT carry an anchor reference; anchoring is out-of-band by design.