Skip to content

daemon: out-of-band checkpoint anchor for tail-truncation resistance#871

Merged
ojongerius merged 4 commits into
mainfrom
claude/checkpoint-anchor-spike-t58tq8
Jun 16, 2026
Merged

daemon: out-of-band checkpoint anchor for tail-truncation resistance#871
ojongerius merged 4 commits into
mainfrom
claude/checkpoint-anchor-spike-t58tq8

Conversation

@ojongerius

Copy link
Copy Markdown
Contributor

Spike (#600) closing the ADR-0008 §2 tail-truncation gap with an additive,
out-of-band signed checkpoint emitted to a pluggable multi-sink anchor.
Receipts stay a linear VC chain — nothing here touches the receipt schema,
hash chain, @context, or issuer DID.

  • checkpoint package: {chain_id, sequence, receipt_hash, timestamp} signed via
    the existing RFC 8785 JCS path; Sign/Verify against the daemon key.
  • Emitter: fan-out to a sink LIST, per-chain cadence (default every receipt) +
    graceful-shutdown flush; sink failures are logged + metered (fail-visible,
    never blocking receipt emission — the opposite of the PY-P9 silent drop).
  • Sinks behind the existing anchor.Sink interface: git (commit chain = the only
    Merkle structure; agent UID cannot write), file (FileLog), syslog (unix).
    OpenSink/OpenSinks dispatch file:/git:/syslog: specs.
  • Daemon wiring: --checkpoint-anchor (fan-out list) + --checkpoint-cadence,
    with env + TOML parity; emit after each commit, flush on shutdown. Empty
    config keeps the commit path byte-identical to before.
  • verify --against-anchor: chain intact + checkpoint signatures + strictly
    increasing log + head-match; a checkpoint ahead of the store HEAD is reported
    FAIL (truncation). Default off — verify is unchanged without the flag.
  • CI gate (ADR-0024 catalogue Rename attest-config- temp dir prefix to ar-config- #11): emit chain, anchor it, truncate the store
    tail, assert verify --against-anchor goes red with a truncation reason while
    plain verify still reports VALID.
  • Freeze: comment at the emit site + a one-line note in ADR-0008 that receipts
    MUST NOT carry an anchor reference; anchoring is out-of-band by design.

Spike (#600) closing the ADR-0008 §2 tail-truncation gap with an additive,
out-of-band signed checkpoint emitted to a pluggable multi-sink anchor.
Receipts stay a linear VC chain — nothing here touches the receipt schema,
hash chain, @context, or issuer DID.

- checkpoint package: {chain_id, sequence, receipt_hash, timestamp} signed via
  the existing RFC 8785 JCS path; Sign/Verify against the daemon key.
- Emitter: fan-out to a sink LIST, per-chain cadence (default every receipt) +
  graceful-shutdown flush; sink failures are logged + metered (fail-visible,
  never blocking receipt emission — the opposite of the PY-P9 silent drop).
- Sinks behind the existing anchor.Sink interface: git (commit chain = the only
  Merkle structure; agent UID cannot write), file (FileLog), syslog (unix).
  OpenSink/OpenSinks dispatch file:/git:/syslog: specs.
- Daemon wiring: --checkpoint-anchor (fan-out list) + --checkpoint-cadence,
  with env + TOML parity; emit after each commit, flush on shutdown. Empty
  config keeps the commit path byte-identical to before.
- verify --against-anchor: chain intact + checkpoint signatures + strictly
  increasing log + head-match; a checkpoint ahead of the store HEAD is reported
  FAIL (truncation). Default off — verify is unchanged without the flag.
- CI gate (ADR-0024 catalogue #11): emit chain, anchor it, truncate the store
  tail, assert verify --against-anchor goes red with a truncation reason while
  plain verify still reports VALID.
- Freeze: comment at the emit site + a one-line note in ADR-0008 that receipts
  MUST NOT carry an anchor reference; anchoring is out-of-band by design.
…reader/PEM parse

Review follow-ups on the checkpoint-anchor spike.

Correctness:
- Emitter now tracks the highest sequence already anchored per chain and gates
  both Observe and Flush on it. The shutdown Flush re-anchored the last head
  whenever the interrupted-chain terminator was skipped (tight/expired deadline),
  writing a second checkpoint at the same sequence — which `verify
  --against-anchor` then rejected as a non-strictly-increasing log, failing a
  healthy, untruncated chain. Flush of an already-anchored head is now a no-op.
- GitLog applies its local config (user identity + commit.gpgsign=false) on every
  open, not only on fresh init, so an operator-provided repo can commit on a host
  that enforces signed commits, and an invalid .git fails loudly at open instead
  of silently on every write.

Cleanup:
- Move the anchor-reading + checkpoint-verify loop into
  checkpoint.ReadVerifiedCheckpoints; verify --against-anchor and the daemon
  end-to-end test now share it (removes a duplicated reader + bufio sizing).
- Export checkpoint.PublicKeyFromPEM and use it from the verify CLI, collapsing
  the two daemon-side Ed25519 PEM parsers into one.

Tests: emitter no-double-emit regression; abrupt-shutdown (1ns deadline) e2e
regression asserting no duplicate checkpoint; pre-existing-repo git config test.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an out-of-band, signed checkpoint anchoring mechanism to the daemon to detect receipt-store tail truncation (ADR-0008 gap), without changing the receipt schema or hash chain. It introduces a checkpoint emitter with pluggable multi-sink anchors (file/git/syslog) and extends obsigna receipt verify with an opt-in --against-anchor check plus CI gates/tests.

Changes:

  • Introduces daemon/internal/checkpoint (checkpoint format, signing/verification, emitter, anchor log reader) and wires it into the pipeline + daemon shutdown flush.
  • Adds anchor sink dispatch and new sink backends (git repo commits, file log reuse, syslog), plus config/flag/env/TOML plumbing.
  • Adds verify --against-anchor and a truncation gate test, plus docs/ADR updates describing the anchoring freeze and verification contract gate.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
docs/adr/0024-project-verification-contract.md Adds Gate #11 for tail-truncation detection via checkpoint anchoring.
docs/adr/0008-response-hashing-and-chain-completeness.md Adds an “anchoring freeze” note: anchoring remains out-of-band and never enters receipts.
daemon/tests_framework.go Adds DaemonFixture.Stop to allow tests to wait for graceful shutdown/flush completion.
daemon/tests_checkpoint_truncation_test.go Adds deterministic truncation gate test that proves --against-anchor detects tail truncation.
daemon/tests_checkpoint_anchor_test.go Adds integration tests covering end-to-end anchoring + shutdown flush and duplicate-avoidance regression.
daemon/README.md Documents checkpoint anchoring, sink specs, cadence, failure behavior, and verify --against-anchor.
daemon/internal/verifycli/verify.go Adds --against-anchor flag and invokes anchor verification after chain validation.
daemon/internal/verifycli/anchor.go Implements the anchor verification logic (head match + truncation detection).
daemon/internal/verifycli/anchor_test.go Unit tests for anchor verification outcomes (pass, truncation, tamper, missing, forged, ordering).
daemon/internal/pipeline/build.go Adds pipeline Checkpointer hook and shutdown FlushCheckpoints.
daemon/internal/checkpoint/reader.go Adds shared anchor-log reader that verifies checkpoint signatures while parsing NDJSON records.
daemon/internal/checkpoint/emitter.go Adds per-chain cadence-based checkpoint emitter with fan-out sinks and failure metering/logging.
daemon/internal/checkpoint/emitter_test.go Unit tests for fan-out, cadence+flush, fail-visible behavior, duplicate prevention, and close.
daemon/internal/checkpoint/checkpoint.go Defines checkpoint payload + signature wrapper and Ed25519 sign/verify helpers.
daemon/internal/checkpoint/checkpoint_test.go Crypto-level tests for sign/verify round-trip and malformed/wrong-key cases.
daemon/internal/anchor/syslog_unix.go Adds syslog sink implementation (unix) using shared record envelope format.
daemon/internal/anchor/syslog_other.go Adds non-unix stub returning a clear “syslog unsupported” error.
daemon/internal/anchor/open.go Adds OpenSink / OpenSinks dispatcher for file/git/syslog sink specs.
daemon/internal/anchor/git.go Adds git-backed sink that appends NDJSON and commits each write for tamper-evident history.
daemon/internal/anchor/git_test.go Tests git sink write/commit behavior, config-on-open, and input validation.
daemon/internal/anchor/anchor.go Adds checkpoint event type and refactors shared record-line marshaling for all sinks.
daemon/daemon.go Wires checkpointing into daemon run lifecycle and flushes checkpoints on shutdown.
daemon/configfile.go Extends TOML config schema with checkpoint anchor list + cadence.
daemon/cmd/obsigna-daemon/main.go Adds flags/env/TOML parsing for checkpoint anchors/cadence and prints resolved config.
daemon/cmd/obsigna-daemon/config_test.go Adds config-layer tests for checkpoint anchor parsing and cadence validation.

Comment thread daemon/internal/verifycli/anchor.go Outdated
Comment thread daemon/internal/checkpoint/checkpoint.go Outdated
Comment thread daemon/internal/anchor/syslog_unix.go
- verify --against-anchor: drop the pre-sort and validate strict monotonicity
  in anchor FILE ORDER. The emitter only ever writes a head past the highest it
  already anchored, so file order is strictly increasing by construction and an
  out-of-order/duplicate record is itself the tamper signal — sorting first
  laundered "seq 2 then seq 1" into a passing "1,2". The last record in file
  order is therefore the genuine latest head. Adds an out-of-order regression
  test.
- checkpoint.Verify: correct the docstring to distinguish (false, err) — the
  artifact could not be evaluated — from (false, nil) — parsed but the signature
  does not match. Behaviour unchanged; only the contract is now accurate.
- syslog sink: strip recordLine's trailing newline before writing, so one
  checkpoint is one syslog message rather than risking a split/multi-line entry.
ojongerius pushed a commit that referenced this pull request Jun 16, 2026
- verify --against-anchor: drop the pre-sort and validate strict monotonicity
  in anchor FILE ORDER. The emitter only ever writes a head past the highest it
  already anchored, so file order is strictly increasing by construction and an
  out-of-order/duplicate record is itself the tamper signal — sorting first
  laundered "seq 2 then seq 1" into a passing "1,2". The last record in file
  order is therefore the genuine latest head. Adds an out-of-order regression
  test.
- checkpoint.Verify: correct the docstring to distinguish (false, err) — the
  artifact could not be evaluated — from (false, nil) — parsed but the signature
  does not match. Behaviour unchanged; only the contract is now accurate.
- syslog sink: strip recordLine's trailing newline before writing, so one
  checkpoint is one syslog message rather than risking a split/multi-line entry.
@ojongerius ojongerius force-pushed the claude/checkpoint-anchor-spike-t58tq8 branch from 290cadd to e02a75a Compare June 16, 2026 23:54
@ojongerius ojongerius merged commit a418625 into main Jun 16, 2026
16 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants