From 52abb3b5c6a2eed7a524459e6ad04bfe04f473c6 Mon Sep 17 00:00:00 2001 From: Otto Jongerius Date: Tue, 16 Jun 2026 21:25:47 +0000 Subject: [PATCH 1/4] daemon: out-of-band checkpoint anchor for tail-truncation resistance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Spike (#600) closing the ADR-0008 §2 tail-truncation gap with an additive, out-of-band signed checkpoint emitted to a pluggable multi-sink anchor. Receipts stay a linear VC chain — nothing here touches the receipt schema, hash chain, @context, or issuer DID. - checkpoint package: {chain_id, sequence, receipt_hash, timestamp} signed via the existing RFC 8785 JCS path; Sign/Verify against the daemon key. - Emitter: fan-out to a sink LIST, per-chain cadence (default every receipt) + graceful-shutdown flush; sink failures are logged + metered (fail-visible, never blocking receipt emission — the opposite of the PY-P9 silent drop). - Sinks behind the existing anchor.Sink interface: git (commit chain = the only Merkle structure; agent UID cannot write), file (FileLog), syslog (unix). OpenSink/OpenSinks dispatch file:/git:/syslog: specs. - Daemon wiring: --checkpoint-anchor (fan-out list) + --checkpoint-cadence, with env + TOML parity; emit after each commit, flush on shutdown. Empty config keeps the commit path byte-identical to before. - verify --against-anchor: chain intact + checkpoint signatures + strictly increasing log + head-match; a checkpoint ahead of the store HEAD is reported FAIL (truncation). Default off — verify is unchanged without the flag. - CI gate (ADR-0024 catalogue #11): emit chain, anchor it, truncate the store tail, assert verify --against-anchor goes red with a truncation reason while plain verify still reports VALID. - Freeze: comment at the emit site + a one-line note in ADR-0008 that receipts MUST NOT carry an anchor reference; anchoring is out-of-band by design. --- daemon/README.md | 42 ++++ daemon/cmd/obsigna-daemon/config_test.go | 65 ++++++ daemon/cmd/obsigna-daemon/main.go | 37 ++++ daemon/configfile.go | 21 +- daemon/daemon.go | 50 +++++ daemon/internal/anchor/anchor.go | 29 ++- daemon/internal/anchor/git.go | 133 +++++++++++ daemon/internal/anchor/git_test.go | 91 ++++++++ daemon/internal/anchor/open.go | 56 +++++ daemon/internal/anchor/syslog_other.go | 12 + daemon/internal/anchor/syslog_unix.go | 65 ++++++ daemon/internal/checkpoint/checkpoint.go | 143 ++++++++++++ daemon/internal/checkpoint/checkpoint_test.go | 122 +++++++++++ daemon/internal/checkpoint/emitter.go | 150 +++++++++++++ daemon/internal/checkpoint/emitter_test.go | 127 +++++++++++ daemon/internal/pipeline/build.go | 64 ++++++ daemon/internal/verifycli/anchor.go | 146 +++++++++++++ daemon/internal/verifycli/anchor_test.go | 175 +++++++++++++++ daemon/internal/verifycli/verify.go | 27 ++- daemon/tests_checkpoint_anchor_test.go | 113 ++++++++++ daemon/tests_checkpoint_truncation_test.go | 206 ++++++++++++++++++ daemon/tests_framework.go | 15 ++ ...response-hashing-and-chain-completeness.md | 2 + .../adr/0024-project-verification-contract.md | 1 + 24 files changed, 1876 insertions(+), 16 deletions(-) create mode 100644 daemon/internal/anchor/git.go create mode 100644 daemon/internal/anchor/git_test.go create mode 100644 daemon/internal/anchor/open.go create mode 100644 daemon/internal/anchor/syslog_other.go create mode 100644 daemon/internal/anchor/syslog_unix.go create mode 100644 daemon/internal/checkpoint/checkpoint.go create mode 100644 daemon/internal/checkpoint/checkpoint_test.go create mode 100644 daemon/internal/checkpoint/emitter.go create mode 100644 daemon/internal/checkpoint/emitter_test.go create mode 100644 daemon/internal/verifycli/anchor.go create mode 100644 daemon/internal/verifycli/anchor_test.go create mode 100644 daemon/tests_checkpoint_anchor_test.go create mode 100644 daemon/tests_checkpoint_truncation_test.go diff --git a/daemon/README.md b/daemon/README.md index b1d0b46e..b1b1b0f1 100644 --- a/daemon/README.md +++ b/daemon/README.md @@ -202,6 +202,48 @@ Exit codes are stable for scripting: | `1` | Chain failed verification (output lists per-receipt status) | | `2` | Usage error (bad flags, missing key file, unreadable DB) | +### Checkpoint anchoring (truncation resistance) + +Chain verification does **not** detect tail truncation on its own (ADR-0008 §2): +dropping the last N receipts leaves a chain that still verifies `VALID`, because +the remaining receipts are internally consistent. The fix is an **out-of-band, +additive checkpoint anchor** — the daemon signs the chain HEAD (`{chain_id, +sequence, receipt_hash, timestamp}`, Ed25519, same RFC 8785 path as receipts) +and writes it to one or more append-only sinks the agent UID cannot rewrite. +Receipts stay a linear chain; the checkpoint is a separate artifact and never +appears in the receipt (the anchoring freeze, ADR-0008). + +Enable it on the daemon: + +```sh +obsigna-daemon \ + --checkpoint-anchor 'git:/var/lib/agentreceipts/anchor,file:/var/log/agentreceipts/anchor.ndjson' \ + --checkpoint-cadence 1 +``` + +- `--checkpoint-anchor` is a comma-separated fan-out list. Each entry is + `git:` (commits to a repo the agent UID cannot write — the commit chain + is the tamper-evident structure), `file:` (append-only NDJSON), or + `syslog:` (local/forwarded syslog, a different host/principal). A bare + path means `file:`. One checkpoint goes to **all** sinks. +- `--checkpoint-cadence` is receipts-per-checkpoint (default `1`, every + receipt). A graceful shutdown always flushes a final checkpoint. +- A sink **write** failure is logged and metered but never blocks or fails + receipt emission — receipts are the primary record, and a missing checkpoint + is caught later by `verify` gap detection. A sink that fails to **open** at + startup is fatal (you asked for an anchor that cannot be provided). + +Check a chain against its anchor (opt-in; default behaviour is unchanged): + +```sh +obsigna receipt verify --against-anchor /var/log/agentreceipts/anchor.ndjson +``` + +This additionally verifies each checkpoint's signature, that the checkpoint log +is strictly increasing, and that the latest checkpoint matches the store HEAD. +A store HEAD *behind* the anchored HEAD is reported as `FAIL (truncation)`. +Without `--against-anchor`, `verify` is byte-identical to today. + ## Read interface: `obsigna receipt show ` ```sh diff --git a/daemon/cmd/obsigna-daemon/config_test.go b/daemon/cmd/obsigna-daemon/config_test.go index e949a8c0..3350049b 100644 --- a/daemon/cmd/obsigna-daemon/config_test.go +++ b/daemon/cmd/obsigna-daemon/config_test.go @@ -50,6 +50,71 @@ shutdown_deadline = "750ms" } } +// TestResolveConfig_CheckpointAnchor covers the checkpoint anchor sink list and +// cadence across all three layers, including the comma-split into a []string. +func TestResolveConfig_CheckpointAnchor(t *testing.T) { + t.Run("file", func(t *testing.T) { + path := writeConfig(t, ` +checkpoint_anchor = "git:/var/anchor, file:/var/a.ndjson" +checkpoint_cadence = 5 +`) + r, err := resolveConfig([]string{"--config", path}, noEnv, io.Discard) + if err != nil { + t.Fatal(err) + } + want := []string{"git:/var/anchor", "file:/var/a.ndjson"} + if len(r.cfg.CheckpointAnchors) != len(want) { + t.Fatalf("anchors = %v, want %v", r.cfg.CheckpointAnchors, want) + } + for i := range want { + if r.cfg.CheckpointAnchors[i] != want[i] { + t.Errorf("anchor[%d] = %q, want %q", i, r.cfg.CheckpointAnchors[i], want[i]) + } + } + if r.cfg.CheckpointCadence != 5 { + t.Errorf("cadence = %d, want 5", r.cfg.CheckpointCadence) + } + }) + t.Run("env overrides file", func(t *testing.T) { + path := writeConfig(t, "checkpoint_anchor = \"file:/from-file\"\n") + env := envMap(map[string]string{"AGENTRECEIPTS_CHECKPOINT_ANCHOR": "git:/from-env"}) + r, err := resolveConfig([]string{"--config", path}, env, io.Discard) + if err != nil { + t.Fatal(err) + } + if len(r.cfg.CheckpointAnchors) != 1 || r.cfg.CheckpointAnchors[0] != "git:/from-env" { + t.Errorf("anchors = %v, want env to override file", r.cfg.CheckpointAnchors) + } + }) + t.Run("flag overrides env and file", func(t *testing.T) { + path := writeConfig(t, "checkpoint_anchor = \"file:/from-file\"\n") + env := envMap(map[string]string{"AGENTRECEIPTS_CHECKPOINT_ANCHOR": "git:/from-env"}) + r, err := resolveConfig([]string{"--config", path, "--checkpoint-anchor", "syslog:tag"}, env, io.Discard) + if err != nil { + t.Fatal(err) + } + if len(r.cfg.CheckpointAnchors) != 1 || r.cfg.CheckpointAnchors[0] != "syslog:tag" { + t.Errorf("anchors = %v, want flag to win", r.cfg.CheckpointAnchors) + } + }) + t.Run("absent means disabled", func(t *testing.T) { + path := writeConfig(t, "chain_id = \"x\"\n") + r, err := resolveConfig([]string{"--config", path}, noEnv, io.Discard) + if err != nil { + t.Fatal(err) + } + if len(r.cfg.CheckpointAnchors) != 0 { + t.Errorf("anchors = %v, want empty (checkpointing off by default)", r.cfg.CheckpointAnchors) + } + }) + t.Run("bad env cadence is rejected", func(t *testing.T) { + env := envMap(map[string]string{"AGENTRECEIPTS_CHECKPOINT_CADENCE": "soon"}) + if _, err := resolveConfig(nil, env, io.Discard); err == nil { + t.Fatal("expected error for non-integer AGENTRECEIPTS_CHECKPOINT_CADENCE") + } + }) +} + // TestResolveConfig_EnvOverridesFile: an env var beats a file value (file is // the lowest-priority layer). func TestResolveConfig_EnvOverridesFile(t *testing.T) { diff --git a/daemon/cmd/obsigna-daemon/main.go b/daemon/cmd/obsigna-daemon/main.go index d9a18fca..8ed5586c 100644 --- a/daemon/cmd/obsigna-daemon/main.go +++ b/daemon/cmd/obsigna-daemon/main.go @@ -223,6 +223,8 @@ func resolveConfig(args []string, getenv func(string) string, errOut io.Writer) fs.BoolVar(&cfg.UnsafeSocketPath, "unsafe-socket-path", cfg.UnsafeSocketPath, "Permit a --socket/AGENTRECEIPTS_SOCKET path outside the per-platform safe set (logs a warning; does not override TCP rejection) (env: AGENTRECEIPTS_UNSAFE_SOCKET_PATH)") fs.StringVar(&cfg.RedactPatternsPath, "redact-patterns", cfg.RedactPatternsPath, "Path to a YAML file of additional redaction patterns (merged with built-in defaults) (env: AGENTRECEIPTS_REDACT_PATTERNS)") fs.DurationVar(&cfg.ShutdownDeadline, "shutdown-deadline", cfg.ShutdownDeadline, "Best-effort time budget for emitting interrupted-chain terminators on SIGTERM/SIGINT (cannot preempt in-progress SQLite I/O)") + checkpointAnchor := fs.String("checkpoint-anchor", strings.Join(cfg.CheckpointAnchors, ","), "Comma-separated out-of-band checkpoint anchor sinks (ADR-0008 truncation anchor): file:, git:, syslog:, or a bare path (=file:). Empty disables checkpointing. (env: AGENTRECEIPTS_CHECKPOINT_ANCHOR)") + fs.IntVar(&cfg.CheckpointCadence, "checkpoint-cadence", cfg.CheckpointCadence, "Receipts between checkpoint emissions per chain; 0/1 = every receipt. A graceful shutdown always forces a final checkpoint. (env: AGENTRECEIPTS_CHECKPOINT_CADENCE)") // scanConfigFlag (via loadConfigLayer) is the source of truth for --config: // it has already consumed the value, so we never read *configPath. The flag @@ -234,6 +236,11 @@ func resolveConfig(args []string, getenv func(string) string, errOut io.Writer) return resolved{}, err } + // --checkpoint-anchor is a comma-separated string on the flag surface but a + // []string in Config; split after Parse so an explicit flag (or its env/ + // default-derived value) lands as the resolved sink list. + cfg.CheckpointAnchors = splitAnchors(*checkpointAnchor) + return resolved{ cfg: cfg, showVersion: *showVersion, @@ -368,6 +375,12 @@ func applyFileConfig(cfg *daemon.Config, fc *daemon.FileConfig) { if fc.ShutdownDeadline != nil { cfg.ShutdownDeadline = fc.ShutdownDeadline.Duration } + if fc.CheckpointAnchor != nil { + cfg.CheckpointAnchors = splitAnchors(*fc.CheckpointAnchor) + } + if fc.CheckpointCadence != nil { + cfg.CheckpointCadence = *fc.CheckpointCadence + } } // envOverlay applies AGENTRECEIPTS_* environment variables over cfg. An unset @@ -418,9 +431,31 @@ func envOverlay(cfg *daemon.Config, getenv func(string) string) error { if v := getenv("AGENTRECEIPTS_REDACT_PATTERNS"); v != "" { cfg.RedactPatternsPath = v } + if v := getenv("AGENTRECEIPTS_CHECKPOINT_ANCHOR"); v != "" { + cfg.CheckpointAnchors = splitAnchors(v) + } + if v := getenv("AGENTRECEIPTS_CHECKPOINT_CADENCE"); v != "" { + n, err := strconv.Atoi(v) + if err != nil { + return fmt.Errorf("invalid AGENTRECEIPTS_CHECKPOINT_CADENCE %q: want an integer", v) + } + cfg.CheckpointCadence = n + } return nil } +// splitAnchors parses a comma-separated checkpoint-anchor list into trimmed, +// non-empty backend specs. "" yields nil (checkpointing disabled). +func splitAnchors(v string) []string { + var specs []string + for _, part := range strings.Split(v, ",") { + if s := strings.TrimSpace(part); s != "" { + specs = append(specs, s) + } + } + return specs +} + // printConfig writes the resolved config in TOML-ish key=value form, mirroring // the config-file keys so the output doubles as a starting daemon.toml. The // signing-key path is printed (it is a filesystem path, not key material — the @@ -438,4 +473,6 @@ func printConfig(w io.Writer, cfg daemon.Config) { fmt.Fprintf(w, "redact_patterns = %q\n", cfg.RedactPatternsPath) fmt.Fprintf(w, "unsafe_socket_path = %t\n", cfg.UnsafeSocketPath) fmt.Fprintf(w, "shutdown_deadline = %q\n", cfg.ShutdownDeadline.String()) + fmt.Fprintf(w, "checkpoint_anchor = %q\n", strings.Join(cfg.CheckpointAnchors, ",")) + fmt.Fprintf(w, "checkpoint_cadence = %d\n", cfg.CheckpointCadence) } diff --git a/daemon/configfile.go b/daemon/configfile.go index 191ae819..995e3900 100644 --- a/daemon/configfile.go +++ b/daemon/configfile.go @@ -19,19 +19,24 @@ import ( // zero value" (e.g. unsafe_socket_path = false) — the config file is the // lowest-priority layer, so an absent key must never clobber env or flags. type FileConfig struct { - Socket *string `toml:"socket"` - DB *string `toml:"db"` - Key *string `toml:"key"` - PublicKey *string `toml:"public_key"` - ForensicPublicKey *string `toml:"forensic_public_key"` - ChainID *string `toml:"chain_id"` - IssuerID *string `toml:"issuer_id"` - VerificationMethod *string `toml:"verification_method"` + Socket *string `toml:"socket"` + DB *string `toml:"db"` + Key *string `toml:"key"` + PublicKey *string `toml:"public_key"` + ForensicPublicKey *string `toml:"forensic_public_key"` + ChainID *string `toml:"chain_id"` + IssuerID *string `toml:"issuer_id"` + VerificationMethod *string `toml:"verification_method"` ParameterDisclosure *DisclosureConfig `toml:"parameter_disclosure"` RedactPatterns *string `toml:"redact_patterns"` UnsafeSocketPath *bool `toml:"unsafe_socket_path"` // ShutdownDeadline accepts a Go duration string, e.g. "200ms" or "1s". ShutdownDeadline *Duration `toml:"shutdown_deadline"` + // CheckpointAnchor mirrors --checkpoint-anchor: a comma-separated list of + // out-of-band checkpoint sink specs (file:/git:/syslog:). CheckpointCadence + // mirrors --checkpoint-cadence. + CheckpointAnchor *string `toml:"checkpoint_anchor"` + CheckpointCadence *int `toml:"checkpoint_cadence"` } // DisclosureConfig is the parsed `parameter_disclosure` config-file value. It diff --git a/daemon/daemon.go b/daemon/daemon.go index 4dd489ae..5fe5bf5d 100644 --- a/daemon/daemon.go +++ b/daemon/daemon.go @@ -19,7 +19,9 @@ import ( "strings" "time" + "github.com/agent-receipts/ar/daemon/internal/anchor" "github.com/agent-receipts/ar/daemon/internal/chain" + "github.com/agent-receipts/ar/daemon/internal/checkpoint" "github.com/agent-receipts/ar/daemon/internal/keysource" "github.com/agent-receipts/ar/daemon/internal/pipeline" "github.com/agent-receipts/ar/daemon/internal/socket" @@ -107,6 +109,24 @@ type Config struct { // ShutdownDeadline is the total time budget for emitting interrupted-chain // terminators on SIGTERM/SIGINT. Defaults to 200ms when zero. ShutdownDeadline time.Duration + + // CheckpointAnchors is the fan-out list of out-of-band sinks the daemon + // writes signed chain-HEAD checkpoints to (ADR-0008 follow-through, the + // truncation anchor). Each entry is a backend spec — "file:", + // "git:", or "syslog:" (a bare path means file). Empty disables + // checkpointing entirely: the daemon's behaviour is then byte-identical to + // a build without this feature, and `verify` without --against-anchor is + // unaffected. Set from --checkpoint-anchor (env: AGENTRECEIPTS_CHECKPOINT_ANCHOR, + // comma-separated). + CheckpointAnchors []string + + // CheckpointCadence is how many committed receipts pass between checkpoint + // emissions on a chain. Defaults to 1 (every receipt) when zero — chosen for + // spike testability; production tuning is deferred. A graceful shutdown + // always forces a final checkpoint regardless of cadence. Only consulted when + // CheckpointAnchors is non-empty. Set from --checkpoint-cadence + // (env: AGENTRECEIPTS_CHECKPOINT_CADENCE). + CheckpointCadence int } // DefaultSocketPath returns the per-OS default socket path. Phase 1 resolves @@ -553,6 +573,31 @@ func Run(ctx context.Context, cfg Config) error { } pp.Redactor = pipeline.NewRedactor(customPatterns) + // Wire the out-of-band checkpoint anchor (ADR-0008 follow-through) when one + // or more sinks are configured. Empty CheckpointAnchors leaves pp.Checkpointer + // nil, so the commit path and shutdown are byte-identical to a build without + // the feature. A sink that fails to OPEN is fatal (the operator asked for an + // anchor that cannot be provided); a sink that fails to WRITE later is + // logged + metered but never blocks receipt emission. + if len(cfg.CheckpointAnchors) > 0 { + sinks, err := anchor.OpenSinks(cfg.CheckpointAnchors) + if err != nil { + return fmt.Errorf("open checkpoint anchor: %w", err) + } + emitter := checkpoint.NewEmitter(sinks, ks, cfg.CheckpointCadence, cfg.Logger.Printf) + defer func() { + if err := emitter.Close(); err != nil { + cfg.Logger.Printf("level=warn checkpoint anchor close: %v", err) + } + }() + pp.Checkpointer = emitter + cadence := cfg.CheckpointCadence + if cadence < 1 { + cadence = 1 + } + cfg.Logger.Printf("checkpoint anchor ACTIVE: %d sink(s), cadence=every %d receipt(s)", len(sinks), cadence) + } + ln, err := socket.Listen(socket.Options{ Path: cfg.SocketPath, Handler: func(ctx context.Context, f socket.Frame) error { return pp.Process(f) }, @@ -583,6 +628,11 @@ func Run(ctx context.Context, cfg Config) error { } } + // Anchor the final HEAD of every open chain (including the terminal receipt + // just emitted) before the deferred st.Close() commits the WAL. No-op when + // checkpointing is disabled. + pp.FlushCheckpoints() + cfg.Logger.Printf("obsigna-daemon shutdown complete") return nil } diff --git a/daemon/internal/anchor/anchor.go b/daemon/internal/anchor/anchor.go index 62b70c5d..3a62b036 100644 --- a/daemon/internal/anchor/anchor.go +++ b/daemon/internal/anchor/anchor.go @@ -25,6 +25,11 @@ import ( // EventTypeRotation is the event type for key_rotated anchor records. const EventTypeRotation = "rotation" +// EventTypeCheckpoint is the event type for signed chain-HEAD checkpoint +// records (the truncation anchor; ADR-0008 follow-through). A sink carries +// both rotation and checkpoint events; readers filter on EventType. +const EventTypeCheckpoint = "checkpoint" + // Sink is an append-only external witness for daemon events. // // Implementations MUST be safe for concurrent use and MUST return an error @@ -71,22 +76,32 @@ func OpenFileLog(path string) (*FileLog, error) { return &FileLog{f: f, now: time.Now}, nil } -// Write appends one JSON record terminated by a newline and fsyncs so the -// record is durable before the caller proceeds to the local commit. -func (l *FileLog) Write(eventType string, payload []byte) error { +// recordLine marshals one newline-terminated anchor Record. anchoredAt is the +// sink-stamped time, so the daemon never chooses the recorded time. Shared by +// every Sink so all adapters emit byte-identical record envelopes. +func recordLine(anchoredAt time.Time, eventType string, payload []byte) ([]byte, error) { if !json.Valid(payload) { - return errors.New("anchor: payload is not valid JSON") + return nil, errors.New("anchor: payload is not valid JSON") } rec := Record{ - AnchoredAt: l.now().UTC().Format(time.RFC3339Nano), + AnchoredAt: anchoredAt.UTC().Format(time.RFC3339Nano), EventType: eventType, Payload: json.RawMessage(payload), } line, err := json.Marshal(rec) if err != nil { - return fmt.Errorf("anchor: marshal record: %w", err) + return nil, fmt.Errorf("anchor: marshal record: %w", err) + } + return append(line, '\n'), nil +} + +// Write appends one JSON record terminated by a newline and fsyncs so the +// record is durable before the caller proceeds to the local commit. +func (l *FileLog) Write(eventType string, payload []byte) error { + line, err := recordLine(l.now(), eventType, payload) + if err != nil { + return err } - line = append(line, '\n') l.mu.Lock() defer l.mu.Unlock() diff --git a/daemon/internal/anchor/git.go b/daemon/internal/anchor/git.go new file mode 100644 index 00000000..e84f8e43 --- /dev/null +++ b/daemon/internal/anchor/git.go @@ -0,0 +1,133 @@ +package anchor + +import ( + "bytes" + "errors" + "fmt" + "os" + "os/exec" + "path/filepath" + "sync" + "time" +) + +// gitCheckpointFile is the tracked, newline-delimited-JSON log inside the git +// anchor repo. Each Write appends one Record line and commits, so the file's +// content is the same format a FileLog produces (verify reads either the +// same way) while the git commit chain adds the tamper-evident layer. +const gitCheckpointFile = "anchor.ndjson" + +// GitLog is an append-only Sink that records each event as a commit in a git +// repository. It is the "different fate-sharing domain" backend of the +// checkpoint seam: the daemon commits to a directory the agent UID cannot +// write (enforced operationally by directory ownership/permissions, the same +// way FileLog's immutability rests on filesystem perms), so an attacker who +// later controls the agent cannot rewrite the anchored history alone. +// +// The git commit chain is the ONLY Merkle structure introduced — receipts stay +// linear (ADR-0008). Each commit's tree head fixes the full checkpoint log up +// to that point, so reordering or dropping an interior checkpoint breaks the +// commit chain. +// +// Implementation note: GitLog shells out to the `git` binary via os/exec +// rather than vendoring a git library — dependency-free and portable across +// macOS and Linux (the daemon's only platforms). git operations are serialised +// under mu; a repo is created on first use if the directory is empty. +type GitLog struct { + mu sync.Mutex + dir string + now func() time.Time +} + +// OpenGitLog opens (initialising if absent) a git anchor repository at dir. +// dir is created with mode 0700 if it does not exist; an existing directory is +// used as-is. The repo is configured with a fixed identity so commits do not +// depend on the host's global git config (which the daemon may not have). +func OpenGitLog(dir string) (*GitLog, error) { + if dir == "" { + return nil, errors.New("anchor: git dir is required") + } + if _, err := exec.LookPath("git"); err != nil { + return nil, fmt.Errorf("anchor: git binary not found: %w", err) + } + if err := os.MkdirAll(dir, 0o700); err != nil { + return nil, fmt.Errorf("anchor: create git dir %s: %w", dir, err) + } + g := &GitLog{dir: dir, now: time.Now} + if _, err := os.Stat(filepath.Join(dir, ".git")); err != nil { + if err := g.run("init", "--quiet"); err != nil { + return nil, fmt.Errorf("anchor: git init %s: %w", dir, err) + } + // Pin a deterministic committer so commits never fail for want of a + // global git identity, and never leak the host's configured one. + if err := g.run("config", "user.email", "daemon@agent-receipts.local"); err != nil { + return nil, fmt.Errorf("anchor: git config user.email: %w", err) + } + if err := g.run("config", "user.name", "obsigna-daemon"); err != nil { + return nil, fmt.Errorf("anchor: git config user.name: %w", err) + } + // Disable commit signing for the anchor repo. Tamper-evidence comes from + // the commit chain and the Ed25519-signed checkpoint payload itself, not + // from a git commit signature — so the anchor must not fail (or stall on a + // passphrase/signing server) just because the host enforces signed commits + // globally. + if err := g.run("config", "commit.gpgsign", "false"); err != nil { + return nil, fmt.Errorf("anchor: git config commit.gpgsign: %w", err) + } + } + return g, nil +} + +// Write appends the record to the tracked log file and commits it. The commit +// is the durable acceptance point: a non-nil error means nothing was committed. +func (g *GitLog) Write(eventType string, payload []byte) error { + line, err := recordLine(g.now(), eventType, payload) + if err != nil { + return err + } + + g.mu.Lock() + defer g.mu.Unlock() + + logPath := filepath.Join(g.dir, gitCheckpointFile) + f, err := os.OpenFile(logPath, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o600) + if err != nil { + return fmt.Errorf("anchor: open git log %s: %w", logPath, err) + } + if _, err := f.Write(line); err != nil { + _ = f.Close() + return fmt.Errorf("anchor: append git log: %w", err) + } + if err := f.Close(); err != nil { + return fmt.Errorf("anchor: close git log: %w", err) + } + + if err := g.run("add", gitCheckpointFile); err != nil { + return fmt.Errorf("anchor: git add: %w", err) + } + msg := fmt.Sprintf("anchor %s @ %s", eventType, g.now().UTC().Format(time.RFC3339Nano)) + if err := g.run("commit", "--quiet", "-m", msg); err != nil { + return fmt.Errorf("anchor: git commit: %w", err) + } + return nil +} + +// Close is a no-op: GitLog holds no long-lived handles (each Write opens and +// closes the log file and runs git fresh). Present to satisfy Sink. +func (g *GitLog) Close() error { return nil } + +// run executes `git ` in the repo directory. Output is captured so a +// failure surfaces git's own diagnostic rather than a bare exit code. +func (g *GitLog) run(args ...string) error { + cmd := exec.Command("git", args...) + cmd.Dir = g.dir + var stderr bytes.Buffer + cmd.Stderr = &stderr + if err := cmd.Run(); err != nil { + if stderr.Len() > 0 { + return fmt.Errorf("%w: %s", err, bytes.TrimSpace(stderr.Bytes())) + } + return err + } + return nil +} diff --git a/daemon/internal/anchor/git_test.go b/daemon/internal/anchor/git_test.go new file mode 100644 index 00000000..60097cb3 --- /dev/null +++ b/daemon/internal/anchor/git_test.go @@ -0,0 +1,91 @@ +package anchor + +import ( + "bufio" + "encoding/json" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" +) + +func TestGitLogCommitsEachWrite(t *testing.T) { + if _, err := exec.LookPath("git"); err != nil { + t.Skip("git not available") + } + dir := filepath.Join(t.TempDir(), "anchor-repo") + g, err := OpenGitLog(dir) + if err != nil { + t.Fatalf("OpenGitLog: %v", err) + } + defer func() { _ = g.Close() }() + + if err := g.Write(EventTypeCheckpoint, []byte(`{"seq":1}`)); err != nil { + t.Fatalf("write 1: %v", err) + } + if err := g.Write(EventTypeCheckpoint, []byte(`{"seq":2}`)); err != nil { + t.Fatalf("write 2: %v", err) + } + + // The tracked log holds both records in the shared anchor.Record format, so + // the verify side reads a git anchor exactly like a file anchor. + f, err := os.Open(filepath.Join(dir, gitCheckpointFile)) + if err != nil { + t.Fatal(err) + } + defer func() { _ = f.Close() }() + var recs []Record + sc := bufio.NewScanner(f) + for sc.Scan() { + var r Record + if err := json.Unmarshal(sc.Bytes(), &r); err != nil { + t.Fatalf("git anchor line is not a Record: %v", err) + } + recs = append(recs, r) + } + if len(recs) != 2 { + t.Fatalf("got %d records, want 2", len(recs)) + } + if recs[0].EventType != EventTypeCheckpoint { + t.Errorf("event_type = %q, want %q", recs[0].EventType, EventTypeCheckpoint) + } + + // Each write is its own commit — the commit chain is the tamper-evident + // Merkle structure (ADR-0008). Two writes => two commits. + out, err := gitOutput(t, dir, "rev-list", "--count", "HEAD") + if err != nil { + t.Fatalf("git rev-list: %v", err) + } + if got := strings.TrimSpace(out); got != "2" { + t.Errorf("commit count = %q, want 2", got) + } +} + +func TestOpenGitLogRequiresDir(t *testing.T) { + if _, err := OpenGitLog(""); err == nil { + t.Fatal("expected error for empty git dir") + } +} + +func TestGitLogRejectsInvalidJSON(t *testing.T) { + if _, err := exec.LookPath("git"); err != nil { + t.Skip("git not available") + } + g, err := OpenGitLog(filepath.Join(t.TempDir(), "repo")) + if err != nil { + t.Fatal(err) + } + defer func() { _ = g.Close() }() + if err := g.Write(EventTypeCheckpoint, []byte("not json")); err == nil { + t.Fatal("expected error for invalid JSON payload") + } +} + +func gitOutput(t *testing.T, dir string, args ...string) (string, error) { + t.Helper() + cmd := exec.Command("git", args...) + cmd.Dir = dir + out, err := cmd.Output() + return string(out), err +} diff --git a/daemon/internal/anchor/open.go b/daemon/internal/anchor/open.go new file mode 100644 index 00000000..dfb7e4fa --- /dev/null +++ b/daemon/internal/anchor/open.go @@ -0,0 +1,56 @@ +package anchor + +import ( + "fmt" + "strings" +) + +// OpenSink constructs a Sink from a backend spec of the form +// ":". The scheme selects the fate-sharing domain; the target +// is scheme-specific: +// +// file: append-only newline-JSON log file (FileLog) +// git: git repository the agent UID cannot write (GitLog) +// syslog: local/forwarded syslog, tag optional (SyslogLog, unix only) +// +// A bare path with no recognised scheme is treated as file: for +// convenience, mirroring how --anchor-log already takes a plain path. +func OpenSink(spec string) (Sink, error) { + spec = strings.TrimSpace(spec) + if spec == "" { + return nil, fmt.Errorf("anchor: empty sink spec") + } + scheme, target, hasScheme := strings.Cut(spec, ":") + if !hasScheme { + // No scheme: treat the whole spec as a file path. + return OpenFileLog(spec) + } + switch scheme { + case "file": + return OpenFileLog(target) + case "git": + return OpenGitLog(target) + case "syslog": + return OpenSyslog(target) + default: + return nil, fmt.Errorf("anchor: unknown sink scheme %q (want file:, git:, or syslog:)", scheme) + } +} + +// OpenSinks constructs the fan-out list from a slice of specs, closing any +// already-opened sinks if a later spec fails so a partial failure does not +// leak file handles or half-initialised repos. +func OpenSinks(specs []string) ([]Sink, error) { + sinks := make([]Sink, 0, len(specs)) + for _, spec := range specs { + s, err := OpenSink(spec) + if err != nil { + for _, opened := range sinks { + _ = opened.Close() + } + return nil, err + } + sinks = append(sinks, s) + } + return sinks, nil +} diff --git a/daemon/internal/anchor/syslog_other.go b/daemon/internal/anchor/syslog_other.go new file mode 100644 index 00000000..654ded4e --- /dev/null +++ b/daemon/internal/anchor/syslog_other.go @@ -0,0 +1,12 @@ +//go:build !unix + +package anchor + +import "errors" + +// OpenSyslog is unavailable off unix (log/syslog is unix-only). The daemon +// itself runs only on linux/darwin, so this branch exists purely to keep the +// package buildable on other platforms for tooling (go vet, cross-compile). +func OpenSyslog(string) (Sink, error) { + return nil, errors.New("anchor: syslog sink is only supported on unix") +} diff --git a/daemon/internal/anchor/syslog_unix.go b/daemon/internal/anchor/syslog_unix.go new file mode 100644 index 00000000..e00cf5d5 --- /dev/null +++ b/daemon/internal/anchor/syslog_unix.go @@ -0,0 +1,65 @@ +//go:build unix + +package anchor + +import ( + "fmt" + "log/syslog" + "sync" + "time" +) + +// SyslogLog is an append-only Sink that writes each record to the local +// syslog daemon. Syslog stands for "a different host/principal" in the +// checkpoint seam: on a host with remote syslog forwarding, the record leaves +// the agent's machine entirely, so truncating the local receipt store cannot +// reach it. This is the deliberately minimal second-domain backend named in +// the spike — it satisfies the same Sink contract as FileLog and GitLog; +// durability/retry tuning is out of scope (#487/#480/#533). +type SyslogLog struct { + mu sync.Mutex + w *syslog.Writer + now func() time.Time +} + +// OpenSyslog connects to the local syslog daemon under tag. An empty tag +// defaults to "obsigna-anchor". A dial failure (no syslog socket) is returned +// so a misconfigured sink fails loudly at open rather than silently dropping +// every checkpoint. +func OpenSyslog(tag string) (*SyslogLog, error) { + if tag == "" { + tag = "obsigna-anchor" + } + w, err := syslog.New(syslog.LOG_NOTICE|syslog.LOG_DAEMON, tag) + if err != nil { + return nil, fmt.Errorf("anchor: connect syslog: %w", err) + } + return &SyslogLog{w: w, now: time.Now}, nil +} + +// Write emits one record line to syslog. A write error means the record was +// not accepted. +func (s *SyslogLog) Write(eventType string, payload []byte) error { + line, err := recordLine(s.now(), eventType, payload) + if err != nil { + return err + } + s.mu.Lock() + defer s.mu.Unlock() + if _, err := s.w.Write(line); err != nil { + return fmt.Errorf("anchor: write syslog: %w", err) + } + return nil +} + +// Close closes the syslog connection. +func (s *SyslogLog) Close() error { + s.mu.Lock() + defer s.mu.Unlock() + if s.w == nil { + return nil + } + err := s.w.Close() + s.w = nil + return err +} diff --git a/daemon/internal/checkpoint/checkpoint.go b/daemon/internal/checkpoint/checkpoint.go new file mode 100644 index 00000000..dcb3cf58 --- /dev/null +++ b/daemon/internal/checkpoint/checkpoint.go @@ -0,0 +1,143 @@ +// Package checkpoint defines the daemon's out-of-band, additive truncation +// anchor (ADR-0008 follow-through; spike per #600). +// +// A checkpoint is a small Ed25519-signed claim about a chain's HEAD — +// {chain_id, sequence, receipt_hash, timestamp} — emitted to one or more +// append-only sinks the agent UID cannot rewrite. Receipts stay a linear +// verifiable-credential chain; the checkpoint is a SEPARATE signed artifact. +// +// IRREVERSIBLE DESIGN CONSTRAINT (ADR-0008): a checkpoint is additive and +// out-of-band. It does NOT touch the receipt schema, the receipt hash chain, +// @context, or the issuer DID — nothing cryptographically bound into receipts +// carries an anchor reference. Anchoring is out-of-band by design, the same +// rationale as the issuer-DID and /context/v1 freezes. If closing the +// truncation gap ever appears to require an in-receipt anchor field, the +// design is wrong — do not move the freeze. +// +// The truncation-detection mechanism this realises is exactly Option B of +// ADR-0008 §3 ("out-of-band commitment"): the verifier obtains the expected +// head out of band (here, the signed checkpoint) and fails when the observed +// chain does not match. The checkpoint payload is canonicalised through the +// EXISTING RFC 8785 JCS path (receipt.Canonicalize) so a checkpoint signs and +// verifies byte-identically to every other signed artifact in the project. +package checkpoint + +import ( + "crypto/ed25519" + "crypto/x509" + "encoding/base64" + "encoding/pem" + "errors" + "fmt" + + "github.com/agent-receipts/ar/sdk/go/receipt" +) + +// multibaseBase64URL mirrors the receipt proof encoding: signatures are +// base64url (multibase prefix "u"), not the W3C default base58btc ("z"). A +// checkpoint signature is therefore decoded the same way a receipt proofValue +// is, so the two cannot drift. +const multibaseBase64URL = "u" + +// Checkpoint is the signed claim about a chain HEAD. The four fields are the +// entire signed body; nothing else is canonicalised. Sequence is the head +// receipt's chain sequence and ReceiptHash is its "sha256:" hash, so a +// verifier can compare both the position and the content of the head it +// observes in the store against the head the daemon last anchored. +type Checkpoint struct { + ChainID string `json:"chain_id"` + Sequence int64 `json:"sequence"` + ReceiptHash string `json:"receipt_hash"` + Timestamp string `json:"timestamp"` +} + +// Signed is a Checkpoint plus its detached signature. It is the artifact +// written to sinks: the embedded Checkpoint fields are the signed body, and +// VerificationMethod/Signature let any reader verify it against the daemon's +// published key without a side channel. The signature covers ONLY the +// Checkpoint body (re-canonicalised on verify), never the wrapper fields — +// identical to how a receipt's proof covers the unsigned receipt, not the +// proof block. +type Signed struct { + Checkpoint + VerificationMethod string `json:"verification_method"` + Signature string `json:"signature"` +} + +// Signer is the minimal signing surface a checkpoint needs. keysource.KeySource +// satisfies it, so checkpoints are signed by the same daemon key as receipts +// without the checkpoint package depending on the daemon's key plumbing. +type Signer interface { + Sign(message []byte) ([]byte, error) + VerificationMethod() string +} + +// Sign canonicalises cp via the RFC 8785 JCS path and signs it, returning the +// Signed artifact ready to hand to a sink. The signature is the raw 64-byte +// Ed25519 form, multibase base64url-encoded to match receipt proofValues. +func Sign(cp Checkpoint, signer Signer) (Signed, error) { + canonical, err := receipt.Canonicalize(cp) + if err != nil { + return Signed{}, fmt.Errorf("canonicalize checkpoint: %w", err) + } + sig, err := signer.Sign([]byte(canonical)) + if err != nil { + return Signed{}, fmt.Errorf("sign checkpoint: %w", err) + } + return Signed{ + Checkpoint: cp, + VerificationMethod: signer.VerificationMethod(), + Signature: multibaseBase64URL + base64.RawURLEncoding.EncodeToString(sig), + }, nil +} + +// Verify checks s's signature against the PEM/SPKI Ed25519 public key. It +// re-canonicalises the embedded Checkpoint body (the wrapper fields are never +// signed) and validates the detached signature. A malformed signature, wrong +// key type, or mismatch returns (false, err) — checkpoint verification failure +// is surfaced, never swallowed. +func Verify(s Signed, publicKeyPEM string) (bool, error) { + if len(s.Signature) < 2 { + return false, errors.New("checkpoint signature too short") + } + if s.Signature[0] != multibaseBase64URL[0] { + return false, fmt.Errorf("unsupported multibase prefix %q (want %q)", s.Signature[0], multibaseBase64URL) + } + sig, err := base64.RawURLEncoding.DecodeString(s.Signature[1:]) + if err != nil { + return false, fmt.Errorf("decode checkpoint signature: %w", err) + } + if len(sig) != ed25519.SignatureSize { + return false, fmt.Errorf("invalid checkpoint signature length: got %d, want %d", len(sig), ed25519.SignatureSize) + } + pub, err := ed25519PublicFromPEM(publicKeyPEM) + if err != nil { + return false, err + } + canonical, err := receipt.Canonicalize(s.Checkpoint) + if err != nil { + return false, fmt.Errorf("canonicalize checkpoint: %w", err) + } + return ed25519.Verify(pub, []byte(canonical), sig), nil +} + +// ed25519PublicFromPEM decodes PEM/SPKI bytes into an Ed25519 public key, +// rejecting any other key type or malformed input. +func ed25519PublicFromPEM(publicKeyPEM string) (ed25519.PublicKey, error) { + block, _ := pem.Decode([]byte(publicKeyPEM)) + if block == nil { + return nil, errors.New("PEM decode failed (no PUBLIC KEY block)") + } + if block.Type != "PUBLIC KEY" { + return nil, fmt.Errorf("PEM block type is %q, want PUBLIC KEY", block.Type) + } + parsed, err := x509.ParsePKIXPublicKey(block.Bytes) + if err != nil { + return nil, fmt.Errorf("parse SPKI public key: %w", err) + } + pub, ok := parsed.(ed25519.PublicKey) + if !ok { + return nil, fmt.Errorf("public key is %T, want ed25519.PublicKey", parsed) + } + return pub, nil +} diff --git a/daemon/internal/checkpoint/checkpoint_test.go b/daemon/internal/checkpoint/checkpoint_test.go new file mode 100644 index 00000000..434a5078 --- /dev/null +++ b/daemon/internal/checkpoint/checkpoint_test.go @@ -0,0 +1,122 @@ +package checkpoint + +import ( + "crypto/ed25519" + "crypto/x509" + "encoding/pem" + "testing" +) + +// testSigner is an in-memory Ed25519 signer satisfying Signer, plus the PEM +// public key for Verify. No file or daemon plumbing — keeps the checkpoint +// crypto tests self-contained. +type testSigner struct { + priv ed25519.PrivateKey + vm string +} + +func (s testSigner) Sign(msg []byte) ([]byte, error) { return ed25519.Sign(s.priv, msg), nil } +func (s testSigner) VerificationMethod() string { return s.vm } + +func newTestSigner(t *testing.T) (testSigner, string) { + t.Helper() + pub, priv, err := ed25519.GenerateKey(nil) + if err != nil { + t.Fatal(err) + } + der, err := x509.MarshalPKIXPublicKey(pub) + if err != nil { + t.Fatal(err) + } + pubPEM := string(pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: der})) + return testSigner{priv: priv, vm: "did:agent-receipts-daemon:test#k1"}, pubPEM +} + +func sampleCheckpoint() Checkpoint { + return Checkpoint{ + ChainID: "2026-06-16", + Sequence: 7, + ReceiptHash: "sha256:" + "ab12", + Timestamp: "2026-06-16T12:00:00Z", + } +} + +func TestSignVerifyRoundTrip(t *testing.T) { + signer, pubPEM := newTestSigner(t) + cp := sampleCheckpoint() + + signed, err := Sign(cp, signer) + if err != nil { + t.Fatalf("Sign: %v", err) + } + if signed.VerificationMethod != signer.vm { + t.Errorf("verification method = %q, want %q", signed.VerificationMethod, signer.vm) + } + if signed.Signature == "" || signed.Signature[0] != 'u' { + t.Errorf("signature %q is not multibase base64url", signed.Signature) + } + // The wrapper must carry the same body it signed. + if signed.Checkpoint != cp { + t.Errorf("signed body = %+v, want %+v", signed.Checkpoint, cp) + } + + ok, err := Verify(signed, pubPEM) + if err != nil { + t.Fatalf("Verify: %v", err) + } + if !ok { + t.Fatal("Verify returned false for a freshly signed checkpoint") + } +} + +func TestVerifyDetectsBodyTamper(t *testing.T) { + signer, pubPEM := newTestSigner(t) + signed, err := Sign(sampleCheckpoint(), signer) + if err != nil { + t.Fatal(err) + } + + // Tamper with the head the checkpoint commits to: a verifier MUST reject it, + // because the signature covers the body, not the wrapper. + signed.ReceiptHash = "sha256:deadbeef" + ok, err := Verify(signed, pubPEM) + if err != nil { + t.Fatalf("Verify: %v", err) + } + if ok { + t.Fatal("Verify accepted a checkpoint whose receipt_hash was altered after signing") + } +} + +func TestVerifyRejectsWrongKey(t *testing.T) { + signer, _ := newTestSigner(t) + _, otherPub := newTestSigner(t) + signed, err := Sign(sampleCheckpoint(), signer) + if err != nil { + t.Fatal(err) + } + ok, err := Verify(signed, otherPub) + if err != nil { + t.Fatalf("Verify: %v", err) + } + if ok { + t.Fatal("Verify accepted a checkpoint under an unrelated public key") + } +} + +func TestVerifyRejectsMalformedSignature(t *testing.T) { + _, pubPEM := newTestSigner(t) + cases := map[string]string{ + "empty": "", + "wrong multibase": "zXYZ", + "bad base64": "u!!!!", + } + for name, sig := range cases { + t.Run(name, func(t *testing.T) { + s := Signed{Checkpoint: sampleCheckpoint(), Signature: sig} + if _, err := Verify(s, pubPEM); err == nil { + t.Errorf("Verify(%q) returned nil error; want a rejection", sig) + } + }) + } +} diff --git a/daemon/internal/checkpoint/emitter.go b/daemon/internal/checkpoint/emitter.go new file mode 100644 index 00000000..3d924297 --- /dev/null +++ b/daemon/internal/checkpoint/emitter.go @@ -0,0 +1,150 @@ +package checkpoint + +import ( + "encoding/json" + "fmt" + "sync" + "sync/atomic" + "time" + + "github.com/agent-receipts/ar/daemon/internal/anchor" +) + +// Emitter signs chain-HEAD checkpoints and fans them out to every configured +// sink. It is the daemon-side glue between the receipt commit path and the +// out-of-band anchor. +// +// Fan-out: one checkpoint goes to ALL sinks (the seam proof — git in one +// fate-sharing domain, file/syslog standing for another). A per-sink write +// failure is logged and counted but NEVER blocks or fails the caller: +// receipts are the primary record, and a missing checkpoint is caught later +// by verify gap-detection. This is the deliberate opposite of the rotation +// anchor's abort-on-failure ordering, and the opposite of the PY-P9 silent- +// drop antipattern — failures are loud (structured log + metric), not fatal +// and not swallowed. +// +// Cadence is "emit once every N observed receipts" per chain (default 1, every +// receipt — chosen for spike testability). Flush forces an emission regardless +// of the counter, used on graceful shutdown. +type Emitter struct { + sinks []anchor.Sink + signer Signer + cadence int + now func() time.Time + logf func(string, ...any) + + mu sync.Mutex + counts map[string]int // observed receipts since last emit, per chain + + emitted atomic.Int64 + failures atomic.Int64 +} + +// NewEmitter returns an Emitter fanning out to sinks, signing with signer. +// cadence < 1 is normalised to 1 (every receipt). logf receives structured +// failure lines; nil silences them (the metric counters still move). +func NewEmitter(sinks []anchor.Sink, signer Signer, cadence int, logf func(string, ...any)) *Emitter { + if cadence < 1 { + cadence = 1 + } + return &Emitter{ + sinks: sinks, + signer: signer, + cadence: cadence, + now: func() time.Time { return time.Now().UTC() }, + logf: logf, + counts: make(map[string]int), + } +} + +// Observe records that chainID advanced to (seq, headHash). When the per-chain +// counter reaches the cadence, a checkpoint is emitted and the counter resets. +// Never returns an error: it runs after the receipt is already committed, so it +// cannot block or undo receipt emission. +func (e *Emitter) Observe(chainID string, seq int64, headHash string) { + e.mu.Lock() + e.counts[chainID]++ + due := e.counts[chainID] >= e.cadence + if due { + e.counts[chainID] = 0 + } + e.mu.Unlock() + + if due { + e.emit(chainID, seq, headHash) + } +} + +// Flush forces a checkpoint for chainID at (seq, headHash) regardless of the +// cadence counter and resets it. Used on graceful shutdown so the final head +// of every open chain is anchored even when the last receipts did not land on +// a cadence boundary. +func (e *Emitter) Flush(chainID string, seq int64, headHash string) { + e.mu.Lock() + e.counts[chainID] = 0 + e.mu.Unlock() + e.emit(chainID, seq, headHash) +} + +// Emitted reports the number of checkpoints written to at least one sink. +// Failures reports the number of (sink, checkpoint) write failures. Both are +// the spike's visibility surface — exposed for tests and a stand-in for the +// real metrics a production emitter would export. +func (e *Emitter) Emitted() int64 { return e.emitted.Load() } +func (e *Emitter) Failures() int64 { return e.failures.Load() } + +// emit signs the checkpoint once and writes it to every sink. A signing +// failure aborts this emission (logged + counted); a sink failure is logged + +// counted but the remaining sinks are still attempted. +func (e *Emitter) emit(chainID string, seq int64, headHash string) { + cp := Checkpoint{ + ChainID: chainID, + Sequence: seq, + ReceiptHash: headHash, + Timestamp: e.now().Format(time.RFC3339Nano), + } + signed, err := Sign(cp, e.signer) + if err != nil { + e.failures.Add(1) + e.log("level=error checkpoint sign failed: chain=%s seq=%d: %v", chainID, seq, err) + return + } + payload, err := json.Marshal(signed) + if err != nil { + e.failures.Add(1) + e.log("level=error checkpoint marshal failed: chain=%s seq=%d: %v", chainID, seq, err) + return + } + + wroteAny := false + for _, s := range e.sinks { + if err := s.Write(anchor.EventTypeCheckpoint, payload); err != nil { + e.failures.Add(1) + e.log("level=error checkpoint sink write failed: chain=%s seq=%d sink=%T: %v", chainID, seq, s, err) + continue + } + wroteAny = true + } + if wroteAny { + e.emitted.Add(1) + } +} + +func (e *Emitter) log(format string, args ...any) { + if e.logf == nil { + return + } + e.logf(format, args...) +} + +// Close closes every sink, returning the first error. Sinks are independent, +// so a failure on one does not stop the others from closing. +func (e *Emitter) Close() error { + var firstErr error + for _, s := range e.sinks { + if err := s.Close(); err != nil && firstErr == nil { + firstErr = fmt.Errorf("close sink %T: %w", s, err) + } + } + return firstErr +} diff --git a/daemon/internal/checkpoint/emitter_test.go b/daemon/internal/checkpoint/emitter_test.go new file mode 100644 index 00000000..eab16db6 --- /dev/null +++ b/daemon/internal/checkpoint/emitter_test.go @@ -0,0 +1,127 @@ +package checkpoint + +import ( + "errors" + "sync" + "testing" + + "github.com/agent-receipts/ar/daemon/internal/anchor" +) + +// recordingSink captures every payload it is asked to write. okWrite toggles +// whether Write succeeds, so a single type covers both the happy and the +// fail-visible paths. +type recordingSink struct { + mu sync.Mutex + writes [][]byte + events []string + okWrite bool + closed bool +} + +func (s *recordingSink) Write(eventType string, payload []byte) error { + s.mu.Lock() + defer s.mu.Unlock() + if !s.okWrite { + return errors.New("sink down") + } + s.events = append(s.events, eventType) + cp := append([]byte(nil), payload...) + s.writes = append(s.writes, cp) + return nil +} + +func (s *recordingSink) Close() error { + s.mu.Lock() + defer s.mu.Unlock() + s.closed = true + return nil +} + +func (s *recordingSink) count() int { + s.mu.Lock() + defer s.mu.Unlock() + return len(s.writes) +} + +func TestEmitterFanOut(t *testing.T) { + signer, _ := newTestSigner(t) + a := &recordingSink{okWrite: true} + b := &recordingSink{okWrite: true} + e := NewEmitter([]anchor.Sink{a, b}, signer, 1, nil) + + e.Observe("chain-1", 1, "sha256:aa") + + if a.count() != 1 || b.count() != 1 { + t.Fatalf("fan-out incomplete: a=%d b=%d, want 1 each", a.count(), b.count()) + } + if got := e.Emitted(); got != 1 { + t.Errorf("Emitted = %d, want 1", got) + } + if got := e.Failures(); got != 0 { + t.Errorf("Failures = %d, want 0", got) + } +} + +func TestEmitterCadence(t *testing.T) { + signer, _ := newTestSigner(t) + sink := &recordingSink{okWrite: true} + e := NewEmitter([]anchor.Sink{sink}, signer, 3, nil) + + // Cadence 3: only the 3rd and 6th observations emit. + for seq := int64(1); seq <= 6; seq++ { + e.Observe("chain-1", seq, "sha256:h") + } + if got := sink.count(); got != 2 { + t.Fatalf("with cadence 3 over 6 receipts, got %d checkpoints, want 2", got) + } + + // Flush forces a final emission regardless of the counter (the graceful- + // shutdown path), so the last head is anchored even off a cadence boundary. + e.Flush("chain-1", 6, "sha256:head") + if got := sink.count(); got != 3 { + t.Fatalf("after Flush, got %d checkpoints, want 3", got) + } +} + +func TestEmitterFailVisibleNotSilent(t *testing.T) { + signer, _ := newTestSigner(t) + down := &recordingSink{okWrite: false} + up := &recordingSink{okWrite: true} + + var logged int + logf := func(string, ...any) { logged++ } + e := NewEmitter([]anchor.Sink{down, up}, signer, 1, logf) + + // One sink fails; the other still receives the checkpoint. The failure is + // counted and logged (visible), and Observe never panics or blocks — it runs + // after the receipt is already committed and must not undo it. + e.Observe("chain-1", 1, "sha256:aa") + + if up.count() != 1 { + t.Errorf("healthy sink got %d writes, want 1 (a failing sibling must not block it)", up.count()) + } + if got := e.Failures(); got != 1 { + t.Errorf("Failures = %d, want 1 (the down sink)", got) + } + if logged == 0 { + t.Error("a sink failure was not logged — failures must be visible, not silent") + } + // At least one sink accepted it, so it counts as emitted. + if got := e.Emitted(); got != 1 { + t.Errorf("Emitted = %d, want 1", got) + } +} + +func TestEmitterClosesAllSinks(t *testing.T) { + signer, _ := newTestSigner(t) + a := &recordingSink{okWrite: true} + b := &recordingSink{okWrite: true} + e := NewEmitter([]anchor.Sink{a, b}, signer, 1, nil) + if err := e.Close(); err != nil { + t.Fatalf("Close: %v", err) + } + if !a.closed || !b.closed { + t.Errorf("Close did not close all sinks: a=%v b=%v", a.closed, b.closed) + } +} diff --git a/daemon/internal/pipeline/build.go b/daemon/internal/pipeline/build.go index 0b555c77..4dbe7e55 100644 --- a/daemon/internal/pipeline/build.go +++ b/daemon/internal/pipeline/build.go @@ -18,6 +18,7 @@ import ( "time" "github.com/agent-receipts/ar/daemon/internal/chain" + "github.com/agent-receipts/ar/daemon/internal/checkpoint" "github.com/agent-receipts/ar/daemon/internal/keysource" "github.com/agent-receipts/ar/daemon/internal/socket" "github.com/agent-receipts/ar/sdk/go/receipt" @@ -180,6 +181,23 @@ type Pipeline struct { // original emitter payload. Nil = no redaction. Redactor *Redactor + // Checkpointer, when set, receives the chain HEAD after every committed + // receipt and emits an out-of-band signed checkpoint per its cadence + // (ADR-0008 follow-through, truncation anchor). Nil = checkpointing + // disabled, which is the default and keeps the commit path byte-identical + // to a build without the feature. + // + // FREEZE (ADR-0008, IRREVERSIBLE): the checkpoint is ADDITIVE and + // OUT-OF-BAND. It is emitted here, AFTER the receipt is committed, from the + // (sequence, hash) the chain already produced — it never reaches back into + // the receipt. Receipts MUST NOT carry an anchor reference; anchoring is + // out-of-band by design (same rationale as the issuer-DID and /context/v1 + // freezes). Do NOT add an anchor field to the receipt schema, the hash + // chain, @context, or the issuer DID to "link" a receipt to its checkpoint. + // If closing the truncation gap appears to need that, the design is wrong — + // the freeze does not move. + Checkpointer *checkpoint.Emitter + // traceMu serialises writes to TraceLog. Process is invoked concurrently // from the listener accept loop, so unguarded fmt.Fprintf calls would // interleave bytes from different frames in the buffer (and race the @@ -400,6 +418,7 @@ func (p *Pipeline) processWithDrop(frame *EmitterFrame, peer socket.PeerCred) er p.trace("receipt stored: seq=%d", liveAlloc.Sequence) liveAlloc.Commit(liveHash) + p.checkpoint(chainID, liveAlloc.Sequence, liveHash) return nil } @@ -434,9 +453,21 @@ func (p *Pipeline) processLive(frame *EmitterFrame, peer socket.PeerCred) error p.trace("receipt stored: seq=%d", alloc.Sequence) alloc.Commit(hash) + p.checkpoint(state.ChainID(), alloc.Sequence, hash) return nil } +// checkpoint hands the just-committed chain HEAD to the Checkpointer, if one is +// configured. It runs AFTER Commit, so it can never block or undo receipt +// emission — a sink failure is logged and metered inside the emitter, not +// surfaced to the frame. No-op when checkpointing is disabled (the default). +func (p *Pipeline) checkpoint(chainID string, seq int64, headHash string) { + if p.Checkpointer == nil { + return + } + p.Checkpointer.Observe(chainID, seq, headHash) +} + // logError calls ErrorLog if it is set. Used for non-fatal conditions where // Process still returns nil (e.g., synthetic receipt failure with live receipt // committed). Callers hold no locks when calling this. @@ -970,6 +1001,39 @@ func (p *Pipeline) EmitTerminator(ctx context.Context) error { return firstErr } +// FlushCheckpoints forces a final checkpoint for every open chain (root and +// per-agent) at its current store HEAD. Called once on graceful shutdown, +// after EmitTerminator, so the anchored head reflects the terminal receipt — +// closing the cadence-boundary gap where the last receipts since the previous +// checkpoint would otherwise be unanchored. No-op when checkpointing is +// disabled. Like the per-receipt path, sink failures are logged/metered inside +// the emitter, never fatal. +func (p *Pipeline) FlushCheckpoints() { + if p.Checkpointer == nil { + return + } + p.agentChainsMu.RLock() + states := make([]*chain.State, 0, len(p.agentChains)+1) + for _, s := range p.agentChains { + states = append(states, s) + } + p.agentChainsMu.RUnlock() + states = append(states, p.State) + + for _, s := range states { + chainID := s.ChainID() + seq, hash, found, err := p.Store.GetChainTail(chainID) + if err != nil { + p.logError("checkpoint flush: read chain tail %s: %v", chainID, err) + continue + } + if !found { + continue + } + p.Checkpointer.Flush(chainID, seq, hash) + } +} + // emitTerminatorForChain emits an interrupted-chain terminal receipt for s if // it has at least one receipt and is not already terminated. func (p *Pipeline) emitTerminatorForChain(ctx context.Context, s *chain.State) error { diff --git a/daemon/internal/verifycli/anchor.go b/daemon/internal/verifycli/anchor.go new file mode 100644 index 00000000..94820d1c --- /dev/null +++ b/daemon/internal/verifycli/anchor.go @@ -0,0 +1,146 @@ +package verifycli + +import ( + "bufio" + "encoding/json" + "fmt" + "os" + "sort" + + "github.com/agent-receipts/ar/daemon/internal/anchor" + "github.com/agent-receipts/ar/daemon/internal/checkpoint" +) + +// anchorResult is the structured outcome of an --against-anchor check. Reason +// is human-readable and empty on success; Truncated marks the specific failure +// the CI verification-contract gate asserts ("no tail truncation"), so a test +// (or a script) can distinguish a truncation from other anchor failures. +type anchorResult struct { + OK bool + Truncated bool + Reason string + Checked int // number of verified checkpoints for the chain +} + +// verifyAgainstAnchor checks the receipt chain HEAD against the out-of-band +// signed checkpoints in the anchor at anchorPath (ADR-0008 follow-through). +// +// It performs, in order: +// - read every anchor record, keep the "checkpoint" events for chainID, and +// verify each one's Ed25519 signature against the published key (pubPEM); +// - assert the checkpoint log is strictly increasing in sequence (an interior +// duplicate/decrease means a corrupted or tampered anchor); +// - assert the latest checkpoint's sequence matches the store HEAD's: a +// checkpoint AHEAD of the store is exactly tail truncation — the anchor +// witnessed receipts the store no longer has (Truncated=true); +// - assert the latest checkpoint's receipt_hash matches the store HEAD's hash +// at that sequence (content tamper at the head). +// +// The caller supplies the store HEAD (seq/hash/found) it already loaded, so the +// anchor check never re-opens the store. +func verifyAgainstAnchor(anchorPath, chainID, pubPEM string, headSeq int64, headHash string, headFound bool) anchorResult { + cps, err := loadCheckpoints(anchorPath, chainID, pubPEM) + if err != nil { + return anchorResult{Reason: err.Error()} + } + if len(cps) == 0 { + return anchorResult{Reason: fmt.Sprintf("no verified checkpoint found for chain %s in anchor %s", chainID, anchorPath)} + } + + sort.Slice(cps, func(i, j int) bool { return cps[i].Sequence < cps[j].Sequence }) + for i := 1; i < len(cps); i++ { + if cps[i].Sequence <= cps[i-1].Sequence { + return anchorResult{ + Checked: len(cps), + Reason: fmt.Sprintf("checkpoint log is not strictly increasing: seq %d follows seq %d", cps[i].Sequence, cps[i-1].Sequence), + } + } + } + latest := cps[len(cps)-1] + + if !headFound { + // The anchor witnessed checkpoints but the store has no receipts for the + // chain — the whole tail (everything up to the anchored head) is gone. + return anchorResult{ + Checked: len(cps), + Truncated: true, + Reason: fmt.Sprintf("anchor records head at seq %d but the store has no receipts for chain %s: chain truncated", latest.Sequence, chainID), + } + } + + switch { + case latest.Sequence > headSeq: + return anchorResult{ + Checked: len(cps), + Truncated: true, + Reason: fmt.Sprintf("anchor records head at seq %d (%s) but store head is seq %d: receipts %d..%d truncated", + latest.Sequence, latest.ReceiptHash, headSeq, headSeq+1, latest.Sequence), + } + case latest.Sequence < headSeq: + return anchorResult{ + Checked: len(cps), + Reason: fmt.Sprintf("store head seq %d is ahead of the latest checkpoint seq %d: checkpoint(s) missing or the store was extended after anchoring", + headSeq, latest.Sequence), + } + case latest.ReceiptHash != headHash: + return anchorResult{ + Checked: len(cps), + Reason: fmt.Sprintf("head receipt hash mismatch at seq %d: anchor has %s, store has %s", + headSeq, latest.ReceiptHash, headHash), + } + } + + return anchorResult{OK: true, Checked: len(cps)} +} + +// loadCheckpoints reads anchorPath, returning the verified checkpoints for +// chainID. A checkpoint whose signature does not verify against pubPEM is a +// hard error — a forged or wrong-key anchor must not be silently skipped. +func loadCheckpoints(anchorPath, chainID, pubPEM string) ([]checkpoint.Checkpoint, error) { + f, err := os.Open(anchorPath) + if err != nil { + return nil, fmt.Errorf("open anchor %s: %w", anchorPath, err) + } + defer func() { _ = f.Close() }() + + var out []checkpoint.Checkpoint + sc := bufio.NewScanner(f) + // Anchor records can be large (a checkpoint payload plus envelope); lift the + // scanner's line cap well above the default 64 KiB so a long line is not + // silently split into a parse error. + sc.Buffer(make([]byte, 0, 64*1024), 4*1024*1024) + line := 0 + for sc.Scan() { + line++ + raw := sc.Bytes() + if len(raw) == 0 { + continue + } + var rec anchor.Record + if err := json.Unmarshal(raw, &rec); err != nil { + return nil, fmt.Errorf("anchor %s line %d: not a JSON record: %w", anchorPath, line, err) + } + if rec.EventType != anchor.EventTypeCheckpoint { + continue + } + var signed checkpoint.Signed + if err := json.Unmarshal(rec.Payload, &signed); err != nil { + return nil, fmt.Errorf("anchor %s line %d: checkpoint payload: %w", anchorPath, line, err) + } + if signed.ChainID != chainID { + continue + } + ok, err := checkpoint.Verify(signed, pubPEM) + if err != nil { + return nil, fmt.Errorf("anchor %s line %d: verify checkpoint (seq %d): %w", anchorPath, line, signed.Sequence, err) + } + if !ok { + return nil, fmt.Errorf("anchor %s line %d: checkpoint signature invalid (seq %d)", anchorPath, line, signed.Sequence) + } + out = append(out, signed.Checkpoint) + } + if err := sc.Err(); err != nil { + return nil, fmt.Errorf("read anchor %s: %w", anchorPath, err) + } + return out, nil +} diff --git a/daemon/internal/verifycli/anchor_test.go b/daemon/internal/verifycli/anchor_test.go new file mode 100644 index 00000000..ca21ad20 --- /dev/null +++ b/daemon/internal/verifycli/anchor_test.go @@ -0,0 +1,175 @@ +package verifycli + +import ( + "crypto/ed25519" + "crypto/x509" + "encoding/json" + "encoding/pem" + "path/filepath" + "strings" + "testing" + + "github.com/agent-receipts/ar/daemon/internal/anchor" + "github.com/agent-receipts/ar/daemon/internal/checkpoint" +) + +type anchorTestSigner struct { + priv ed25519.PrivateKey +} + +func (s anchorTestSigner) Sign(msg []byte) ([]byte, error) { return ed25519.Sign(s.priv, msg), nil } +func (s anchorTestSigner) VerificationMethod() string { return "did:test#k1" } + +func newAnchorSigner(t *testing.T) (anchorTestSigner, string) { + t.Helper() + pub, priv, err := ed25519.GenerateKey(nil) + if err != nil { + t.Fatal(err) + } + der, err := x509.MarshalPKIXPublicKey(pub) + if err != nil { + t.Fatal(err) + } + return anchorTestSigner{priv: priv}, string(pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: der})) +} + +// writeAnchor builds a real anchor file (via the shared FileLog format) holding +// the given checkpoints, each signed by signer. Returns the file path. +func writeAnchor(t *testing.T, signer checkpoint.Signer, cps ...checkpoint.Checkpoint) string { + t.Helper() + path := filepath.Join(t.TempDir(), "anchor.ndjson") + log, err := anchor.OpenFileLog(path) + if err != nil { + t.Fatal(err) + } + defer func() { _ = log.Close() }() + for _, cp := range cps { + signed, err := checkpoint.Sign(cp, signer) + if err != nil { + t.Fatal(err) + } + payload, err := json.Marshal(signed) + if err != nil { + t.Fatal(err) + } + if err := log.Write(anchor.EventTypeCheckpoint, payload); err != nil { + t.Fatal(err) + } + } + return path +} + +func cp(chainID string, seq int64, hash string) checkpoint.Checkpoint { + return checkpoint.Checkpoint{ChainID: chainID, Sequence: seq, ReceiptHash: hash, Timestamp: "2026-06-16T00:00:00Z"} +} + +func TestAnchorPassesWhenHeadMatches(t *testing.T) { + signer, pub := newAnchorSigner(t) + path := writeAnchor(t, signer, + cp("c", 1, "sha256:a"), cp("c", 2, "sha256:b"), cp("c", 3, "sha256:head")) + + got := verifyAgainstAnchor(path, "c", pub, 3, "sha256:head", true) + if !got.OK { + t.Fatalf("expected PASS, got fail: %s", got.Reason) + } + if got.Checked != 3 { + t.Errorf("Checked = %d, want 3", got.Checked) + } +} + +func TestAnchorDetectsTailTruncation(t *testing.T) { + signer, pub := newAnchorSigner(t) + // Anchor witnessed 5 receipts; the store now only has 3 (tail truncated). + path := writeAnchor(t, signer, + cp("c", 1, "sha256:1"), cp("c", 2, "sha256:2"), cp("c", 3, "sha256:3"), + cp("c", 4, "sha256:4"), cp("c", 5, "sha256:5")) + + got := verifyAgainstAnchor(path, "c", pub, 3, "sha256:3", true) + if got.OK { + t.Fatal("expected FAIL for a truncated tail, got PASS") + } + if !got.Truncated { + t.Errorf("expected Truncated=true, got reason: %s", got.Reason) + } + if !strings.Contains(got.Reason, "truncat") { + t.Errorf("reason %q does not mention truncation", got.Reason) + } +} + +func TestAnchorDetectsWholeChainGone(t *testing.T) { + signer, pub := newAnchorSigner(t) + path := writeAnchor(t, signer, cp("c", 1, "sha256:1"), cp("c", 2, "sha256:2")) + + got := verifyAgainstAnchor(path, "c", pub, 0, "", false) + if got.OK || !got.Truncated { + t.Fatalf("expected truncation FAIL when store is empty but anchor has checkpoints; got %+v", got) + } +} + +func TestAnchorDetectsHeadHashTamper(t *testing.T) { + signer, pub := newAnchorSigner(t) + path := writeAnchor(t, signer, cp("c", 1, "sha256:1"), cp("c", 2, "sha256:realhead")) + + // Same sequence, different head hash → tamper, not truncation. + got := verifyAgainstAnchor(path, "c", pub, 2, "sha256:forged", true) + if got.OK { + t.Fatal("expected FAIL for head-hash mismatch") + } + if got.Truncated { + t.Error("hash mismatch at equal sequence is tamper, not truncation") + } + if !strings.Contains(got.Reason, "mismatch") { + t.Errorf("reason %q does not mention mismatch", got.Reason) + } +} + +func TestAnchorDetectsStoreAheadOfAnchor(t *testing.T) { + signer, pub := newAnchorSigner(t) + path := writeAnchor(t, signer, cp("c", 1, "sha256:1"), cp("c", 2, "sha256:2")) + + got := verifyAgainstAnchor(path, "c", pub, 5, "sha256:5", true) + if got.OK { + t.Fatal("expected FAIL when the store head is ahead of the latest checkpoint") + } + if got.Truncated { + t.Error("store-ahead is not a truncation") + } +} + +func TestAnchorRejectsForgedCheckpoint(t *testing.T) { + signer, _ := newAnchorSigner(t) + _, otherPub := newAnchorSigner(t) // verify against a DIFFERENT key + path := writeAnchor(t, signer, cp("c", 1, "sha256:1")) + + got := verifyAgainstAnchor(path, "c", otherPub, 1, "sha256:1", true) + if got.OK { + t.Fatal("expected FAIL when checkpoint signature does not verify against the published key") + } +} + +func TestAnchorNoCheckpointForChain(t *testing.T) { + signer, pub := newAnchorSigner(t) + path := writeAnchor(t, signer, cp("other-chain", 1, "sha256:1")) + + got := verifyAgainstAnchor(path, "c", pub, 1, "sha256:1", true) + if got.OK { + t.Fatal("expected FAIL when no checkpoint exists for the chain") + } + if !strings.Contains(got.Reason, "no verified checkpoint") { + t.Errorf("reason %q unexpected", got.Reason) + } +} + +func TestAnchorRejectsNonMonotonicLog(t *testing.T) { + signer, pub := newAnchorSigner(t) + // Duplicate sequence — a corrupted or tampered anchor log. + path := writeAnchor(t, signer, cp("c", 1, "sha256:1"), cp("c", 1, "sha256:dup")) + + got := verifyAgainstAnchor(path, "c", pub, 1, "sha256:1", true) + if got.OK { + t.Fatal("expected FAIL for a non-monotonic checkpoint log") + } + if !strings.Contains(got.Reason, "increasing") { + t.Errorf("reason %q does not flag the ordering problem", got.Reason) + } +} diff --git a/daemon/internal/verifycli/verify.go b/daemon/internal/verifycli/verify.go index f251b78a..1576473a 100644 --- a/daemon/internal/verifycli/verify.go +++ b/daemon/internal/verifycli/verify.go @@ -75,6 +75,7 @@ func Run(args []string, stdout, stderr io.Writer, envLookup func(string) string) dbPath := fs.String("db", envOr("AGENTRECEIPTS_DB", daemon.DefaultDBPath()), "SQLite receipt-store path (env: AGENTRECEIPTS_DB)") pubKeyPath := fs.String("public-key", defaultPubKey, "PEM-encoded SPKI public key path (env: AGENTRECEIPTS_PUBLIC_KEY)") chainID := fs.String("chain-id", envOr("AGENTRECEIPTS_CHAIN_ID", time.Now().UTC().Format("2006-01-02")), "Chain id to verify (env: AGENTRECEIPTS_CHAIN_ID)") + againstAnchor := fs.String("against-anchor", envOr("AGENTRECEIPTS_AGAINST_ANCHOR", ""), "Path to an out-of-band checkpoint anchor log (ADR-0008). When set, verify additionally checks the chain HEAD against the latest signed checkpoint and fails on tail truncation. Default off: behaviour without this flag is unchanged. (env: AGENTRECEIPTS_AGAINST_ANCHOR)") if err := fs.Parse(args); err != nil { // `-h` / `--help` is intentional, not an error — flag.ContinueOnError // surfaces it as flag.ErrHelp after writing the usage message. Exit 0 @@ -197,7 +198,31 @@ func Run(args []string, stdout, stderr io.Writer, envLookup func(string) string) noun = "receipt" } fmt.Fprintf(stdout, "Chain %s: VALID (%d %s)\n", *chainID, result.Length, noun) - return ExitOK + + // Out-of-band anchor check (ADR-0008), opt-in via --against-anchor. A + // tail-truncated chain still verifies as VALID above — its remaining + // receipts are internally consistent — so the anchor is the only thing + // that can catch a dropped tail. Default off: without the flag this + // returns here, byte-identical to before. + if *againstAnchor == "" { + return ExitOK + } + headSeq, headHash, headFound, terr := s.GetChainTail(*chainID) + if terr != nil { + fmt.Fprintf(stderr, "obsigna receipt verify: read chain tail for anchor check: %v\n", terr) + return ExitUsageError + } + ar := verifyAgainstAnchor(*againstAnchor, *chainID, string(pubPEM), headSeq, headHash, headFound) + if ar.OK { + fmt.Fprintf(stdout, "Anchor %s: PASS (%d checkpoint(s); head seq %d anchored)\n", *againstAnchor, ar.Checked, headSeq) + return ExitOK + } + label := "FAIL" + if ar.Truncated { + label = "FAIL (truncation)" + } + fmt.Fprintf(stdout, "Anchor %s: %s — %s\n", *againstAnchor, label, ar.Reason) + return ExitChainBad } fmt.Fprintf(stdout, "Chain %s: BROKEN at receipt %d\n", *chainID, result.BrokenAt) if result.Error != "" { diff --git a/daemon/tests_checkpoint_anchor_test.go b/daemon/tests_checkpoint_anchor_test.go new file mode 100644 index 00000000..13c5284d --- /dev/null +++ b/daemon/tests_checkpoint_anchor_test.go @@ -0,0 +1,113 @@ +//go:build integration && (linux || darwin) + +package daemon + +import ( + "bufio" + "encoding/json" + "os" + "path/filepath" + "testing" + "time" + + "github.com/agent-receipts/ar/daemon/internal/anchor" + "github.com/agent-receipts/ar/daemon/internal/checkpoint" + "github.com/agent-receipts/ar/sdk/go/store" +) + +// TestCheckpointAnchorEndToEnd exercises the full daemon wiring: a Config with +// a checkpoint anchor sink starts the daemon, real emitter frames flow over the +// socket, the daemon signs a checkpoint per receipt to the out-of-band anchor, +// and after a graceful shutdown (which flushes the final HEAD) the anchored +// head is a verifiable checkpoint that matches the store HEAD. This is the +// Config→Run→OpenSinks→emitter→FlushCheckpoints→Close path the deterministic +// pipeline-level gate (tests_checkpoint_truncation_test.go) does not cover. +func TestCheckpointAnchorEndToEnd(t *testing.T) { + cfg, pubPEM := newDaemonConfig(t, 0) + anchorPath := filepath.Join(t.TempDir(), "anchor.ndjson") + cfg.CheckpointAnchors = []string{"file:" + anchorPath} + cfg.CheckpointCadence = 1 + + fix := StartDaemonFromConfig(t, cfg, pubPEM) + + const n = 3 + for i := 0; i < n; i++ { + if err := fix.EmitGoFrame(t, "sess-cp", "mcp_proxy", "list_repos", "github", "allowed"); err != nil { + t.Fatalf("emit %d: %v", i, err) + } + } + fix.WaitForReceiptCount(t, n, 2*time.Second) + + // Graceful shutdown: drains the listener, emits the interrupted-chain + // terminator, flushes a final checkpoint for the terminal head, and closes + // the sinks. + fix.Stop(t) + + cps := readVerifiedCheckpoints(t, anchorPath, cfg.ChainID, pubPEM) + if len(cps) == 0 { + t.Fatal("no verified checkpoints in the anchor — the daemon did not anchor anything") + } + + // The latest checkpoint must match the store HEAD after a clean shutdown + // (the flush anchors the terminal receipt). + latest := cps[len(cps)-1] + s, err := store.OpenReadOnly(cfg.DBPath) + if err != nil { + t.Fatalf("open store: %v", err) + } + defer func() { _ = s.Close() }() + seq, hash, found, err := s.GetChainTail(cfg.ChainID) + if err != nil { + t.Fatalf("get chain tail: %v", err) + } + if !found { + t.Fatal("store has no receipts") + } + if latest.Sequence != seq { + t.Errorf("latest checkpoint seq = %d, store head seq = %d", latest.Sequence, seq) + } + if latest.ReceiptHash != hash { + t.Errorf("latest checkpoint hash = %s, store head hash = %s", latest.ReceiptHash, hash) + } +} + +// readVerifiedCheckpoints reads the anchor file and returns the checkpoints for +// chainID whose signatures verify against pubPEM, in file order. A signature +// that does not verify fails the test — the daemon must sign with its own key. +func readVerifiedCheckpoints(t *testing.T, path, chainID, pubPEM string) []checkpoint.Checkpoint { + t.Helper() + f, err := os.Open(path) + if err != nil { + t.Fatalf("open anchor: %v", err) + } + defer func() { _ = f.Close() }() + + var out []checkpoint.Checkpoint + sc := bufio.NewScanner(f) + sc.Buffer(make([]byte, 0, 64*1024), 4*1024*1024) + for sc.Scan() { + var rec anchor.Record + if err := json.Unmarshal(sc.Bytes(), &rec); err != nil { + t.Fatalf("anchor line not a Record: %v", err) + } + if rec.EventType != anchor.EventTypeCheckpoint { + continue + } + var signed checkpoint.Signed + if err := json.Unmarshal(rec.Payload, &signed); err != nil { + t.Fatalf("checkpoint payload: %v", err) + } + if signed.ChainID != chainID { + continue + } + ok, err := checkpoint.Verify(signed, pubPEM) + if err != nil || !ok { + t.Fatalf("checkpoint (seq %d) failed verification: ok=%v err=%v", signed.Sequence, ok, err) + } + out = append(out, signed.Checkpoint) + } + if err := sc.Err(); err != nil { + t.Fatalf("scan anchor: %v", err) + } + return out +} diff --git a/daemon/tests_checkpoint_truncation_test.go b/daemon/tests_checkpoint_truncation_test.go new file mode 100644 index 00000000..61dcf90c --- /dev/null +++ b/daemon/tests_checkpoint_truncation_test.go @@ -0,0 +1,206 @@ +package daemon_test + +import ( + "bytes" + "crypto/ed25519" + "crypto/x509" + "database/sql" + "encoding/json" + "encoding/pem" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/agent-receipts/ar/daemon/internal/anchor" + "github.com/agent-receipts/ar/daemon/internal/chain" + "github.com/agent-receipts/ar/daemon/internal/checkpoint" + "github.com/agent-receipts/ar/daemon/internal/keysource" + "github.com/agent-receipts/ar/daemon/internal/pipeline" + "github.com/agent-receipts/ar/daemon/internal/socket" + "github.com/agent-receipts/ar/daemon/internal/verifycli" + "github.com/agent-receipts/ar/sdk/go/store" + + _ "modernc.org/sqlite" +) + +// TestCheckpointAnchorCatchesTailTruncation is the verification-contract gate +// for ADR-0024 ("No tail truncation"): emit a chain, anchor a signed checkpoint +// per receipt, truncate the receipt-store tail, then run `verify +// --against-anchor` and assert it goes RED with a truncation reason. The same +// chain, verified WITHOUT the flag, must still report VALID — proving both that +// truncation is invisible to plain chain verification (the gap ADR-0008 names) +// and that the anchor closes it. +// +// Driven through the pipeline directly (no socket) so the gate is deterministic +// and fast; the daemon Config→Run→emitter wiring is covered separately by the +// integration test. +func TestCheckpointAnchorCatchesTailTruncation(t *testing.T) { + const ( + chainID = "trunc-chain" + issuerID = "did:agent-receipts-daemon:test" + vm = issuerID + "#k1" + total = 5 + keep = 3 // truncate the store back to this many receipts + ) + + dir := t.TempDir() + dbPath := filepath.Join(dir, "receipts.db") + keyPath, pubPath := writeTestKeypair(t, dir, vm) + anchorPath := filepath.Join(dir, "anchor.ndjson") + + // --- emit + anchor ----------------------------------------------------- + st, err := store.Open(dbPath) + if err != nil { + t.Fatalf("open store: %v", err) + } + ks := keysource.NewFile(keyPath, vm) + if err := ks.Init(); err != nil { + t.Fatalf("keysource init: %v", err) + } + sink, err := anchor.OpenSink("file:" + anchorPath) + if err != nil { + t.Fatalf("open sink: %v", err) + } + emitter := checkpoint.NewEmitter([]anchor.Sink{sink}, ks, 1, nil) + + pp := pipeline.New(chain.New(chainID), ks, st, issuerID) + pp.Checkpointer = emitter + + for i := 0; i < total; i++ { + if err := pp.Process(sampleCheckpointFrame(t)); err != nil { + t.Fatalf("process frame %d: %v", i, err) + } + } + if got := emitter.Emitted(); got != total { + t.Fatalf("emitted %d checkpoints, want %d", got, total) + } + if got := emitter.Failures(); got != 0 { + t.Fatalf("emitter reported %d failures, want 0", got) + } + if err := emitter.Close(); err != nil { + t.Fatalf("close emitter: %v", err) + } + if err := st.Close(); err != nil { + t.Fatalf("close store: %v", err) + } + + // --- happy path: head matches the latest checkpoint ------------------- + if code, out := runVerify(t, dbPath, pubPath, chainID, anchorPath); code != verifycli.ExitOK { + t.Fatalf("pre-truncation verify exit = %d, want %d\n%s", code, verifycli.ExitOK, out) + } else if !strings.Contains(out, "PASS") { + t.Fatalf("pre-truncation anchor check did not PASS:\n%s", out) + } + + // --- truncate the receipt-store tail ---------------------------------- + truncateChainTail(t, dbPath, chainID, keep) + + // Without the flag, a tail-truncated chain still verifies VALID: the + // remaining receipts are internally consistent. This is the byte-identical + // default behaviour the freeze guarantees. + if code, out := runVerifyNoAnchor(t, dbPath, pubPath, chainID); code != verifycli.ExitOK { + t.Fatalf("plain verify of truncated chain exit = %d, want %d (default behaviour must be unchanged)\n%s", code, verifycli.ExitOK, out) + } else if !strings.Contains(out, "VALID") { + t.Fatalf("plain verify of truncated chain should still report VALID:\n%s", out) + } + + // With --against-anchor, the dropped tail is caught and verification fails. + code, out := runVerify(t, dbPath, pubPath, chainID, anchorPath) + if code != verifycli.ExitChainBad { + t.Fatalf("post-truncation verify exit = %d, want %d (truncation must fail)\n%s", code, verifycli.ExitChainBad, out) + } + if !strings.Contains(strings.ToLower(out), "truncat") { + t.Fatalf("post-truncation verify did not give a truncation reason:\n%s", out) + } +} + +// sampleCheckpointFrame builds a minimal valid live emitter frame. +func sampleCheckpointFrame(t *testing.T) socket.Frame { + t.Helper() + body, err := json.Marshal(pipeline.EmitterFrame{ + Version: "1", + TsEmit: time.Now().UTC().Format(time.RFC3339), + SessionID: "sess-trunc", + Channel: "mcp_proxy", + Tool: pipeline.EmitterTool{Server: "github", Name: "list_repos"}, + Decision: "allowed", + }) + if err != nil { + t.Fatal(err) + } + return socket.Frame{ + Payload: body, + Peer: socket.PeerCred{ + Platform: "linux", + PID: 1234, + UID: 1000, + GID: 1000, + }, + } +} + +func writeTestKeypair(t *testing.T, dir, _ string) (keyPath, pubPath string) { + t.Helper() + pub, priv, err := ed25519.GenerateKey(nil) + if err != nil { + t.Fatal(err) + } + privDER, err := x509.MarshalPKCS8PrivateKey(priv) + if err != nil { + t.Fatal(err) + } + pubDER, err := x509.MarshalPKIXPublicKey(pub) + if err != nil { + t.Fatal(err) + } + keyPath = filepath.Join(dir, "signing.key") + pubPath = filepath.Join(dir, "signing.key.pub") + if err := os.WriteFile(keyPath, pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: privDER}), 0o600); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(pubPath, pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: pubDER}), 0o644); err != nil { + t.Fatal(err) + } + return keyPath, pubPath +} + +// truncateChainTail deletes every receipt in chainID with sequence > keep, +// standing in for an attacker who drops the tail of the receipt store. +func truncateChainTail(t *testing.T, dbPath, chainID string, keep int) { + t.Helper() + db, err := sql.Open("sqlite", dbPath) + if err != nil { + t.Fatalf("open db for truncation: %v", err) + } + defer func() { _ = db.Close() }() + res, err := db.Exec("DELETE FROM receipts WHERE chain_id = ? AND sequence > ?", chainID, keep) + if err != nil { + t.Fatalf("truncate: %v", err) + } + n, _ := res.RowsAffected() + if n == 0 { + t.Fatal("truncation deleted 0 rows — test setup is wrong") + } +} + +func runVerify(t *testing.T, dbPath, pubPath, chainID, anchorPath string) (int, string) { + t.Helper() + return runVerifyArgs(t, []string{ + "--db", dbPath, "--public-key", pubPath, "--chain-id", chainID, "--against-anchor", anchorPath, + }) +} + +func runVerifyNoAnchor(t *testing.T, dbPath, pubPath, chainID string) (int, string) { + t.Helper() + return runVerifyArgs(t, []string{ + "--db", dbPath, "--public-key", pubPath, "--chain-id", chainID, + }) +} + +func runVerifyArgs(t *testing.T, args []string) (int, string) { + t.Helper() + var stdout, stderr bytes.Buffer + code := verifycli.Run(args, &stdout, &stderr, func(string) string { return "" }) + return code, stdout.String() + stderr.String() +} diff --git a/daemon/tests_framework.go b/daemon/tests_framework.go index ae69b2dc..7eefc361 100644 --- a/daemon/tests_framework.go +++ b/daemon/tests_framework.go @@ -427,6 +427,21 @@ func (f *DaemonFixture) Trace() string { return f.traceBuf.String() } +// Stop triggers a graceful shutdown and waits for Run to return, so a test can +// inspect on-disk state (store, anchor) after the daemon's deferred cleanup — +// including the shutdown checkpoint flush — has run. Idempotent with the +// t.Cleanup that also cancels: the second cancel is a no-op and done is already +// closed. +func (f *DaemonFixture) Stop(t *testing.T) { + t.Helper() + f.cancel() + select { + case <-f.done: + case <-time.After(3 * time.Second): + t.Fatal("daemon did not shut down within 3s") + } +} + // findSDKRoot locates the repo root by walking up from the current working // directory until it finds the repo-root go.work file. The monorepo always // has a committed go.work (see /AGENTS.md "Go workspace"), so falling back diff --git a/docs/adr/0008-response-hashing-and-chain-completeness.md b/docs/adr/0008-response-hashing-and-chain-completeness.md index 3fe4d252..2b05b9cb 100644 --- a/docs/adr/0008-response-hashing-and-chain-completeness.md +++ b/docs/adr/0008-response-hashing-and-chain-completeness.md @@ -55,6 +55,8 @@ The release that closes #153 and #171 ships four things together: Option C (in-chain length-commitment field) is rejected as security theatre — see Context §"Options considered". Transparency-log-style checkpointing remains deferred to a future ADR. +> **Anchoring freeze (added by the checkpoint-anchor spike, #600).** Receipts MUST NOT carry an anchor reference; anchoring is out-of-band by design — same rationale as the issuer-DID and `/context/v1` freezes. The signed checkpoint that realises Option B's "out-of-band commitment" (see §3) is a separate, additive artifact emitted to a pluggable multi-sink anchor; it never touches the receipt schema, the hash chain, `@context`, or the issuer DID. If closing the truncation gap appears to require an in-receipt anchor field, the design is wrong — the freeze does not move. + ### 1. Response hashing (from #153) Add an optional field `credentialSubject.outcome.response_hash` containing the SHA-256 hash of the RFC 8785 canonical JSON of the server's response, computed **after** secret redaction. diff --git a/docs/adr/0024-project-verification-contract.md b/docs/adr/0024-project-verification-contract.md index 5e4d28fe..0d3a5284 100644 --- a/docs/adr/0024-project-verification-contract.md +++ b/docs/adr/0024-project-verification-contract.md @@ -50,6 +50,7 @@ The following gates are committed to as the initial set, each tracked by its own | 8 | **Daemon ↔ SDK protocol compatibility** | An SDK release declares a daemon-protocol version range and the gate verifies the released daemon at the same time speaks a compatible range | #655 | | 9 | **Spec source-of-truth integrity** | At most one canonical spec source file exists at a time; every spec version's `@context` URL exists in the repo at `spec/context/v/context.jsonld` | #597 follow-through | | 10 | **Documented dependencies match installed dependencies** | The dependencies the README claims match an SBOM produced at release time; unexplained eager dependencies fail the build | #656 | +| 11 | **No tail truncation (checkpoint anchor)** | A chain with an out-of-band checkpoint anchor cannot have its receipt-store tail truncated undetected: `verify --against-anchor` fails when the anchored HEAD is ahead of the store HEAD | #600 | ### D5. Gate exemptions are explicit From 808b05bd1d7517ddfabc328e7b1681be6913d047 Mon Sep 17 00:00:00 2001 From: Otto Jongerius Date: Tue, 16 Jun 2026 23:23:05 +0000 Subject: [PATCH 2/4] daemon: fix duplicate shutdown checkpoint + git config; dedup anchor reader/PEM parse MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-ups on the checkpoint-anchor spike. Correctness: - Emitter now tracks the highest sequence already anchored per chain and gates both Observe and Flush on it. The shutdown Flush re-anchored the last head whenever the interrupted-chain terminator was skipped (tight/expired deadline), writing a second checkpoint at the same sequence — which `verify --against-anchor` then rejected as a non-strictly-increasing log, failing a healthy, untruncated chain. Flush of an already-anchored head is now a no-op. - GitLog applies its local config (user identity + commit.gpgsign=false) on every open, not only on fresh init, so an operator-provided repo can commit on a host that enforces signed commits, and an invalid .git fails loudly at open instead of silently on every write. Cleanup: - Move the anchor-reading + checkpoint-verify loop into checkpoint.ReadVerifiedCheckpoints; verify --against-anchor and the daemon end-to-end test now share it (removes a duplicated reader + bufio sizing). - Export checkpoint.PublicKeyFromPEM and use it from the verify CLI, collapsing the two daemon-side Ed25519 PEM parsers into one. Tests: emitter no-double-emit regression; abrupt-shutdown (1ns deadline) e2e regression asserting no duplicate checkpoint; pre-existing-repo git config test. --- daemon/internal/anchor/git.go | 37 ++++++----- daemon/internal/anchor/git_test.go | 33 ++++++++++ daemon/internal/checkpoint/checkpoint.go | 11 ++-- daemon/internal/checkpoint/emitter.go | 34 ++++++++-- daemon/internal/checkpoint/emitter_test.go | 48 +++++++++++--- daemon/internal/checkpoint/reader.go | 66 +++++++++++++++++++ daemon/internal/verifycli/anchor.go | 58 +---------------- daemon/internal/verifycli/verify.go | 30 +-------- daemon/tests_checkpoint_anchor_test.go | 73 ++++++++++------------ 9 files changed, 231 insertions(+), 159 deletions(-) create mode 100644 daemon/internal/checkpoint/reader.go diff --git a/daemon/internal/anchor/git.go b/daemon/internal/anchor/git.go index e84f8e43..bffddc1a 100644 --- a/daemon/internal/anchor/git.go +++ b/daemon/internal/anchor/git.go @@ -58,21 +58,28 @@ func OpenGitLog(dir string) (*GitLog, error) { if err := g.run("init", "--quiet"); err != nil { return nil, fmt.Errorf("anchor: git init %s: %w", dir, err) } - // Pin a deterministic committer so commits never fail for want of a - // global git identity, and never leak the host's configured one. - if err := g.run("config", "user.email", "daemon@agent-receipts.local"); err != nil { - return nil, fmt.Errorf("anchor: git config user.email: %w", err) - } - if err := g.run("config", "user.name", "obsigna-daemon"); err != nil { - return nil, fmt.Errorf("anchor: git config user.name: %w", err) - } - // Disable commit signing for the anchor repo. Tamper-evidence comes from - // the commit chain and the Ed25519-signed checkpoint payload itself, not - // from a git commit signature — so the anchor must not fail (or stall on a - // passphrase/signing server) just because the host enforces signed commits - // globally. - if err := g.run("config", "commit.gpgsign", "false"); err != nil { - return nil, fmt.Errorf("anchor: git config commit.gpgsign: %w", err) + } + // Apply the anchor repo's local config on every open, not only on fresh + // init: an operator may point --checkpoint-anchor at a repo they created + // themselves (which would otherwise lack these settings), and the settings + // are idempotent. Running them unconditionally also turns an existing but + // invalid .git (a stray file, a corrupt repo) into a loud failure here at + // startup — "sink fails to open is fatal" — instead of a silent per-write + // failure on every checkpoint. + // + // user.name/email pin a deterministic committer so commits never fail for + // want of a global identity (and never leak the host's). commit.gpgsign + // false is load-bearing: tamper-evidence comes from the commit chain and the + // Ed25519-signed checkpoint payload, not a git signature, so the anchor must + // not fail or stall on a passphrase/signing server when the host enforces + // signed commits globally. + for _, kv := range [][2]string{ + {"user.email", "daemon@agent-receipts.local"}, + {"user.name", "obsigna-daemon"}, + {"commit.gpgsign", "false"}, + } { + if err := g.run("config", kv[0], kv[1]); err != nil { + return nil, fmt.Errorf("anchor: git config %s in %s: %w", kv[0], dir, err) } } return g, nil diff --git a/daemon/internal/anchor/git_test.go b/daemon/internal/anchor/git_test.go index 60097cb3..a805334b 100644 --- a/daemon/internal/anchor/git_test.go +++ b/daemon/internal/anchor/git_test.go @@ -68,6 +68,39 @@ func TestOpenGitLogRequiresDir(t *testing.T) { } } +// TestGitLogConfiguresPreExistingRepo guards the fix for the case where an +// operator points --checkpoint-anchor at a repo they created themselves: the +// repo lacks the daemon's local identity and commit.gpgsign=false, so without +// applying config on every open, the first commit would fail on a host that +// enforces signed commits. OpenGitLog must configure an existing repo too. +func TestGitLogConfiguresPreExistingRepo(t *testing.T) { + if _, err := exec.LookPath("git"); err != nil { + t.Skip("git not available") + } + dir := filepath.Join(t.TempDir(), "preexisting") + if err := os.MkdirAll(dir, 0o700); err != nil { + t.Fatal(err) + } + // Pre-create the repo WITHOUT the daemon's config (mirrors an operator's + // `git init`). On a host enforcing commit signing globally this repo cannot + // commit until commit.gpgsign=false is set locally. + preInit := exec.Command("git", "init", "--quiet") + preInit.Dir = dir + if err := preInit.Run(); err != nil { + t.Fatalf("pre-init: %v", err) + } + + g, err := OpenGitLog(dir) + if err != nil { + t.Fatalf("OpenGitLog on existing repo: %v", err) + } + defer func() { _ = g.Close() }() + + if err := g.Write(EventTypeCheckpoint, []byte(`{"seq":1}`)); err != nil { + t.Fatalf("write to pre-existing repo failed (config not applied?): %v", err) + } +} + func TestGitLogRejectsInvalidJSON(t *testing.T) { if _, err := exec.LookPath("git"); err != nil { t.Skip("git not available") diff --git a/daemon/internal/checkpoint/checkpoint.go b/daemon/internal/checkpoint/checkpoint.go index dcb3cf58..f679d59b 100644 --- a/daemon/internal/checkpoint/checkpoint.go +++ b/daemon/internal/checkpoint/checkpoint.go @@ -110,7 +110,7 @@ func Verify(s Signed, publicKeyPEM string) (bool, error) { if len(sig) != ed25519.SignatureSize { return false, fmt.Errorf("invalid checkpoint signature length: got %d, want %d", len(sig), ed25519.SignatureSize) } - pub, err := ed25519PublicFromPEM(publicKeyPEM) + pub, err := PublicKeyFromPEM([]byte(publicKeyPEM)) if err != nil { return false, err } @@ -121,10 +121,11 @@ func Verify(s Signed, publicKeyPEM string) (bool, error) { return ed25519.Verify(pub, []byte(canonical), sig), nil } -// ed25519PublicFromPEM decodes PEM/SPKI bytes into an Ed25519 public key, -// rejecting any other key type or malformed input. -func ed25519PublicFromPEM(publicKeyPEM string) (ed25519.PublicKey, error) { - block, _ := pem.Decode([]byte(publicKeyPEM)) +// PublicKeyFromPEM decodes PEM/SPKI bytes into an Ed25519 public key, rejecting +// any other key type or malformed input. Shared by Verify and the verify CLI's +// anchor check so the daemon has a single Ed25519 public-key parser. +func PublicKeyFromPEM(publicKeyPEM []byte) (ed25519.PublicKey, error) { + block, _ := pem.Decode(publicKeyPEM) if block == nil { return nil, errors.New("PEM decode failed (no PUBLIC KEY block)") } diff --git a/daemon/internal/checkpoint/emitter.go b/daemon/internal/checkpoint/emitter.go index 3d924297..f8527a17 100644 --- a/daemon/internal/checkpoint/emitter.go +++ b/daemon/internal/checkpoint/emitter.go @@ -33,8 +33,9 @@ type Emitter struct { now func() time.Time logf func(string, ...any) - mu sync.Mutex - counts map[string]int // observed receipts since last emit, per chain + mu sync.Mutex + counts map[string]int // observed receipts since last emit, per chain + lastSeq map[string]int64 // highest sequence already anchored, per chain emitted atomic.Int64 failures atomic.Int64 @@ -54,6 +55,7 @@ func NewEmitter(sinks []anchor.Sink, signer Signer, cadence int, logf func(strin now: func() time.Time { return time.Now().UTC() }, logf: logf, counts: make(map[string]int), + lastSeq: make(map[string]int64), } } @@ -64,13 +66,14 @@ func NewEmitter(sinks []anchor.Sink, signer Signer, cadence int, logf func(strin func (e *Emitter) Observe(chainID string, seq int64, headHash string) { e.mu.Lock() e.counts[chainID]++ - due := e.counts[chainID] >= e.cadence - if due { + emit := false + if e.counts[chainID] >= e.cadence { e.counts[chainID] = 0 + emit = e.markEmitLocked(chainID, seq) } e.mu.Unlock() - if due { + if emit { e.emit(chainID, seq, headHash) } } @@ -82,8 +85,27 @@ func (e *Emitter) Observe(chainID string, seq int64, headHash string) { func (e *Emitter) Flush(chainID string, seq int64, headHash string) { e.mu.Lock() e.counts[chainID] = 0 + emit := e.markEmitLocked(chainID, seq) e.mu.Unlock() - e.emit(chainID, seq, headHash) + if emit { + e.emit(chainID, seq, headHash) + } +} + +// markEmitLocked reports whether a checkpoint at seq should be emitted for +// chainID, and records it as anchored when so. It gates on the highest sequence +// already anchored: a head at or below it has been anchored already, so +// re-emitting would write a duplicate (or out-of-order) checkpoint that +// verify reads as a non-strictly-increasing — i.e. corrupt — log. This is what +// makes Flush idempotent against the per-receipt Observe path: a graceful +// shutdown whose terminator was skipped (deadline) re-flushes the last head, +// and without this guard that head would be anchored twice. Caller holds e.mu. +func (e *Emitter) markEmitLocked(chainID string, seq int64) bool { + if seq <= e.lastSeq[chainID] { + return false + } + e.lastSeq[chainID] = seq + return true } // Emitted reports the number of checkpoints written to at least one sink. diff --git a/daemon/internal/checkpoint/emitter_test.go b/daemon/internal/checkpoint/emitter_test.go index eab16db6..c83206c1 100644 --- a/daemon/internal/checkpoint/emitter_test.go +++ b/daemon/internal/checkpoint/emitter_test.go @@ -68,19 +68,21 @@ func TestEmitterCadence(t *testing.T) { sink := &recordingSink{okWrite: true} e := NewEmitter([]anchor.Sink{sink}, signer, 3, nil) - // Cadence 3: only the 3rd and 6th observations emit. - for seq := int64(1); seq <= 6; seq++ { + // Cadence 3 over 5 receipts: only the 3rd observation emits (seqs 4,5 leave + // the counter at 2, below the cadence). + for seq := int64(1); seq <= 5; seq++ { e.Observe("chain-1", seq, "sha256:h") } - if got := sink.count(); got != 2 { - t.Fatalf("with cadence 3 over 6 receipts, got %d checkpoints, want 2", got) + if got := sink.count(); got != 1 { + t.Fatalf("with cadence 3 over 5 receipts, got %d checkpoints, want 1", got) } - // Flush forces a final emission regardless of the counter (the graceful- - // shutdown path), so the last head is anchored even off a cadence boundary. - e.Flush("chain-1", 6, "sha256:head") - if got := sink.count(); got != 3 { - t.Fatalf("after Flush, got %d checkpoints, want 3", got) + // Flush forces a final emission of the current head (seq 5) even though it + // fell off a cadence boundary and was not yet anchored (the graceful-shutdown + // path). seq 5 > the last-anchored seq 3, so it emits. + e.Flush("chain-1", 5, "sha256:head") + if got := sink.count(); got != 2 { + t.Fatalf("after Flush of an unanchored head, got %d checkpoints, want 2", got) } } @@ -113,6 +115,34 @@ func TestEmitterFailVisibleNotSilent(t *testing.T) { } } +func TestEmitterFlushDoesNotReEmitAlreadyAnchoredHead(t *testing.T) { + signer, _ := newTestSigner(t) + sink := &recordingSink{okWrite: true} + e := NewEmitter([]anchor.Sink{sink}, signer, 1, nil) + + // Per-receipt Observe anchors head seq 3 (cadence 1). + e.Observe("c", 3, "sha256:h3") + if sink.count() != 1 { + t.Fatalf("after Observe(3): got %d checkpoints, want 1", sink.count()) + } + + // Graceful shutdown re-flushes the SAME head (e.g. the terminator was skipped + // on a tight deadline, so the tail is still seq 3). This must NOT write a + // second checkpoint at seq 3 — a duplicate makes verify read the anchor log + // as non-strictly-increasing and fail a healthy chain (the bug this guards). + e.Flush("c", 3, "sha256:h3") + if sink.count() != 1 { + t.Fatalf("Flush re-anchored an already-anchored head: got %d checkpoints, want 1", sink.count()) + } + + // A flush at a genuinely new head (the terminator advanced the chain to 4) + // still anchors. + e.Flush("c", 4, "sha256:h4") + if sink.count() != 2 { + t.Fatalf("Flush at a new head did not anchor: got %d checkpoints, want 2", sink.count()) + } +} + func TestEmitterClosesAllSinks(t *testing.T) { signer, _ := newTestSigner(t) a := &recordingSink{okWrite: true} diff --git a/daemon/internal/checkpoint/reader.go b/daemon/internal/checkpoint/reader.go new file mode 100644 index 00000000..a4c44f4f --- /dev/null +++ b/daemon/internal/checkpoint/reader.go @@ -0,0 +1,66 @@ +package checkpoint + +import ( + "bufio" + "encoding/json" + "fmt" + "os" + + "github.com/agent-receipts/ar/daemon/internal/anchor" +) + +// ReadVerifiedCheckpoints reads an anchor log at path and returns the +// checkpoints recorded for chainID whose signatures verify against the +// PEM/SPKI public key publicKeyPEM, in file order. Records for other event +// types or other chains are skipped. A checkpoint whose signature does NOT +// verify is a hard error — a forged or wrong-key anchor must not be silently +// dropped. This is the single anchor reader shared by `verify --against-anchor` +// and the daemon's end-to-end test, so both parse the anchor identically. +func ReadVerifiedCheckpoints(path, chainID string, publicKeyPEM string) ([]Checkpoint, error) { + f, err := os.Open(path) + if err != nil { + return nil, fmt.Errorf("open anchor %s: %w", path, err) + } + defer func() { _ = f.Close() }() + + var out []Checkpoint + sc := bufio.NewScanner(f) + // Anchor records can be large (a checkpoint payload plus envelope); lift the + // scanner's line cap well above the default 64 KiB so a long line is not + // silently split into a parse error. + sc.Buffer(make([]byte, 0, 64*1024), 4*1024*1024) + lineNo := 0 + for sc.Scan() { + lineNo++ + raw := sc.Bytes() + if len(raw) == 0 { + continue + } + var rec anchor.Record + if err := json.Unmarshal(raw, &rec); err != nil { + return nil, fmt.Errorf("anchor %s line %d: not a JSON record: %w", path, lineNo, err) + } + if rec.EventType != anchor.EventTypeCheckpoint { + continue + } + var signed Signed + if err := json.Unmarshal(rec.Payload, &signed); err != nil { + return nil, fmt.Errorf("anchor %s line %d: checkpoint payload: %w", path, lineNo, err) + } + if signed.ChainID != chainID { + continue + } + ok, err := Verify(signed, publicKeyPEM) + if err != nil { + return nil, fmt.Errorf("anchor %s line %d: verify checkpoint (seq %d): %w", path, lineNo, signed.Sequence, err) + } + if !ok { + return nil, fmt.Errorf("anchor %s line %d: checkpoint signature invalid (seq %d)", path, lineNo, signed.Sequence) + } + out = append(out, signed.Checkpoint) + } + if err := sc.Err(); err != nil { + return nil, fmt.Errorf("read anchor %s: %w", path, err) + } + return out, nil +} diff --git a/daemon/internal/verifycli/anchor.go b/daemon/internal/verifycli/anchor.go index 94820d1c..61eb7f8e 100644 --- a/daemon/internal/verifycli/anchor.go +++ b/daemon/internal/verifycli/anchor.go @@ -1,13 +1,9 @@ package verifycli import ( - "bufio" - "encoding/json" "fmt" - "os" "sort" - "github.com/agent-receipts/ar/daemon/internal/anchor" "github.com/agent-receipts/ar/daemon/internal/checkpoint" ) @@ -39,7 +35,7 @@ type anchorResult struct { // The caller supplies the store HEAD (seq/hash/found) it already loaded, so the // anchor check never re-opens the store. func verifyAgainstAnchor(anchorPath, chainID, pubPEM string, headSeq int64, headHash string, headFound bool) anchorResult { - cps, err := loadCheckpoints(anchorPath, chainID, pubPEM) + cps, err := checkpoint.ReadVerifiedCheckpoints(anchorPath, chainID, pubPEM) if err != nil { return anchorResult{Reason: err.Error()} } @@ -92,55 +88,3 @@ func verifyAgainstAnchor(anchorPath, chainID, pubPEM string, headSeq int64, head return anchorResult{OK: true, Checked: len(cps)} } - -// loadCheckpoints reads anchorPath, returning the verified checkpoints for -// chainID. A checkpoint whose signature does not verify against pubPEM is a -// hard error — a forged or wrong-key anchor must not be silently skipped. -func loadCheckpoints(anchorPath, chainID, pubPEM string) ([]checkpoint.Checkpoint, error) { - f, err := os.Open(anchorPath) - if err != nil { - return nil, fmt.Errorf("open anchor %s: %w", anchorPath, err) - } - defer func() { _ = f.Close() }() - - var out []checkpoint.Checkpoint - sc := bufio.NewScanner(f) - // Anchor records can be large (a checkpoint payload plus envelope); lift the - // scanner's line cap well above the default 64 KiB so a long line is not - // silently split into a parse error. - sc.Buffer(make([]byte, 0, 64*1024), 4*1024*1024) - line := 0 - for sc.Scan() { - line++ - raw := sc.Bytes() - if len(raw) == 0 { - continue - } - var rec anchor.Record - if err := json.Unmarshal(raw, &rec); err != nil { - return nil, fmt.Errorf("anchor %s line %d: not a JSON record: %w", anchorPath, line, err) - } - if rec.EventType != anchor.EventTypeCheckpoint { - continue - } - var signed checkpoint.Signed - if err := json.Unmarshal(rec.Payload, &signed); err != nil { - return nil, fmt.Errorf("anchor %s line %d: checkpoint payload: %w", anchorPath, line, err) - } - if signed.ChainID != chainID { - continue - } - ok, err := checkpoint.Verify(signed, pubPEM) - if err != nil { - return nil, fmt.Errorf("anchor %s line %d: verify checkpoint (seq %d): %w", anchorPath, line, signed.Sequence, err) - } - if !ok { - return nil, fmt.Errorf("anchor %s line %d: checkpoint signature invalid (seq %d)", anchorPath, line, signed.Sequence) - } - out = append(out, signed.Checkpoint) - } - if err := sc.Err(); err != nil { - return nil, fmt.Errorf("read anchor %s: %w", anchorPath, err) - } - return out, nil -} diff --git a/daemon/internal/verifycli/verify.go b/daemon/internal/verifycli/verify.go index 1576473a..146bb9fe 100644 --- a/daemon/internal/verifycli/verify.go +++ b/daemon/internal/verifycli/verify.go @@ -16,11 +16,8 @@ package verifycli import ( - "crypto/ed25519" "crypto/sha256" - "crypto/x509" "encoding/hex" - "encoding/pem" "errors" "flag" "fmt" @@ -31,6 +28,7 @@ import ( "time" "github.com/agent-receipts/ar/daemon" + "github.com/agent-receipts/ar/daemon/internal/checkpoint" "github.com/agent-receipts/ar/sdk/go/receipt" "github.com/agent-receipts/ar/sdk/go/store" ) @@ -252,38 +250,16 @@ func Run(args []string, stdout, stderr io.Writer, envLookup func(string) string) // malformed key would surface as a "BROKEN" chain — falsely implicating the // receipts. func validatePublicKeyPEM(pubPEM []byte) error { - _, err := ed25519PublicFromPEM(pubPEM) + _, err := checkpoint.PublicKeyFromPEM(pubPEM) return err } -// ed25519PublicFromPEM decodes PEM/SPKI bytes into an Ed25519 public key, -// rejecting any other key type or malformed input. It is the single parse behind -// both validatePublicKeyPEM and publicKeyFingerprint so the two cannot diverge. -func ed25519PublicFromPEM(pubPEM []byte) (ed25519.PublicKey, error) { - block, _ := pem.Decode(pubPEM) - if block == nil { - return nil, errors.New("PEM decode failed (no PUBLIC KEY block)") - } - if block.Type != "PUBLIC KEY" { - return nil, fmt.Errorf("PEM block type is %q, want PUBLIC KEY", block.Type) - } - parsed, err := x509.ParsePKIXPublicKey(block.Bytes) - if err != nil { - return nil, fmt.Errorf("parse SPKI public key: %w", err) - } - pub, ok := parsed.(ed25519.PublicKey) - if !ok { - return nil, fmt.Errorf("public key is %T, want ed25519.PublicKey", parsed) - } - return pub, nil -} - // publicKeyFingerprint returns the ADR-0015 fingerprint of a PEM/SPKI Ed25519 // public key: SHA-256 of the raw 32-byte key, as sha256:. This // matches the construction the rotation writer and the SDK use, so it compares // directly against a key_rotated receipt's new_key_fingerprint. func publicKeyFingerprint(pubPEM []byte) (string, error) { - pub, err := ed25519PublicFromPEM(pubPEM) + pub, err := checkpoint.PublicKeyFromPEM(pubPEM) if err != nil { return "", err } diff --git a/daemon/tests_checkpoint_anchor_test.go b/daemon/tests_checkpoint_anchor_test.go index 13c5284d..0e5e1196 100644 --- a/daemon/tests_checkpoint_anchor_test.go +++ b/daemon/tests_checkpoint_anchor_test.go @@ -3,14 +3,10 @@ package daemon import ( - "bufio" - "encoding/json" - "os" "path/filepath" "testing" "time" - "github.com/agent-receipts/ar/daemon/internal/anchor" "github.com/agent-receipts/ar/daemon/internal/checkpoint" "github.com/agent-receipts/ar/sdk/go/store" ) @@ -43,7 +39,10 @@ func TestCheckpointAnchorEndToEnd(t *testing.T) { // the sinks. fix.Stop(t) - cps := readVerifiedCheckpoints(t, anchorPath, cfg.ChainID, pubPEM) + cps, err := checkpoint.ReadVerifiedCheckpoints(anchorPath, cfg.ChainID, pubPEM) + if err != nil { + t.Fatalf("read anchor: %v", err) + } if len(cps) == 0 { t.Fatal("no verified checkpoints in the anchor — the daemon did not anchor anything") } @@ -71,43 +70,37 @@ func TestCheckpointAnchorEndToEnd(t *testing.T) { } } -// readVerifiedCheckpoints reads the anchor file and returns the checkpoints for -// chainID whose signatures verify against pubPEM, in file order. A signature -// that does not verify fails the test — the daemon must sign with its own key. -func readVerifiedCheckpoints(t *testing.T, path, chainID, pubPEM string) []checkpoint.Checkpoint { - t.Helper() - f, err := os.Open(path) - if err != nil { - t.Fatalf("open anchor: %v", err) - } - defer func() { _ = f.Close() }() +// TestCheckpointAnchorAbruptShutdownNoDuplicate is the regression for the +// shutdown-flush double-emit bug. With a 1ns shutdown deadline the +// interrupted-chain terminator is skipped, so the chain tail stays at the last +// live receipt — the head the per-receipt Observe already anchored. The +// shutdown FlushCheckpoints must NOT re-anchor that head: a duplicate checkpoint +// at the same sequence makes `verify --against-anchor` read the log as +// non-strictly-increasing and fail a perfectly healthy chain. +func TestCheckpointAnchorAbruptShutdownNoDuplicate(t *testing.T) { + cfg, pubPEM := newDaemonConfig(t, time.Nanosecond) // crash-grade deadline → terminator skipped + anchorPath := filepath.Join(t.TempDir(), "anchor.ndjson") + cfg.CheckpointAnchors = []string{"file:" + anchorPath} + cfg.CheckpointCadence = 1 - var out []checkpoint.Checkpoint - sc := bufio.NewScanner(f) - sc.Buffer(make([]byte, 0, 64*1024), 4*1024*1024) - for sc.Scan() { - var rec anchor.Record - if err := json.Unmarshal(sc.Bytes(), &rec); err != nil { - t.Fatalf("anchor line not a Record: %v", err) - } - if rec.EventType != anchor.EventTypeCheckpoint { - continue - } - var signed checkpoint.Signed - if err := json.Unmarshal(rec.Payload, &signed); err != nil { - t.Fatalf("checkpoint payload: %v", err) - } - if signed.ChainID != chainID { - continue - } - ok, err := checkpoint.Verify(signed, pubPEM) - if err != nil || !ok { - t.Fatalf("checkpoint (seq %d) failed verification: ok=%v err=%v", signed.Sequence, ok, err) + fix := StartDaemonFromConfig(t, cfg, pubPEM) + const n = 3 + for i := 0; i < n; i++ { + if err := fix.EmitGoFrame(t, "sess-cp", "mcp_proxy", "list_repos", "github", "allowed"); err != nil { + t.Fatalf("emit %d: %v", i, err) } - out = append(out, signed.Checkpoint) } - if err := sc.Err(); err != nil { - t.Fatalf("scan anchor: %v", err) + fix.WaitForReceiptCount(t, n, 2*time.Second) + fix.Stop(t) + + cps, err := checkpoint.ReadVerifiedCheckpoints(anchorPath, cfg.ChainID, pubPEM) + if err != nil { + t.Fatalf("read anchor: %v", err) + } + for i := 1; i < len(cps); i++ { + if cps[i].Sequence <= cps[i-1].Sequence { + t.Fatalf("duplicate/out-of-order checkpoint after abrupt shutdown: seq %d follows seq %d (all: %+v)", + cps[i].Sequence, cps[i-1].Sequence, cps) + } } - return out } From dcd9c9ecf92ba7e5c279f9fd7700cb99435f68f2 Mon Sep 17 00:00:00 2001 From: Otto Jongerius Date: Tue, 16 Jun 2026 23:34:16 +0000 Subject: [PATCH 3/4] daemon: address Copilot review on checkpoint anchor (#871) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - verify --against-anchor: drop the pre-sort and validate strict monotonicity in anchor FILE ORDER. The emitter only ever writes a head past the highest it already anchored, so file order is strictly increasing by construction and an out-of-order/duplicate record is itself the tamper signal — sorting first laundered "seq 2 then seq 1" into a passing "1,2". The last record in file order is therefore the genuine latest head. Adds an out-of-order regression test. - checkpoint.Verify: correct the docstring to distinguish (false, err) — the artifact could not be evaluated — from (false, nil) — parsed but the signature does not match. Behaviour unchanged; only the contract is now accurate. - syslog sink: strip recordLine's trailing newline before writing, so one checkpoint is one syslog message rather than risking a split/multi-line entry. --- daemon/internal/anchor/syslog_unix.go | 10 ++++++++-- daemon/internal/checkpoint/checkpoint.go | 14 +++++++++++--- daemon/internal/verifycli/anchor.go | 9 +++++++-- daemon/internal/verifycli/anchor_test.go | 16 ++++++++++++++++ 4 files changed, 42 insertions(+), 7 deletions(-) diff --git a/daemon/internal/anchor/syslog_unix.go b/daemon/internal/anchor/syslog_unix.go index e00cf5d5..7805954a 100644 --- a/daemon/internal/anchor/syslog_unix.go +++ b/daemon/internal/anchor/syslog_unix.go @@ -3,6 +3,7 @@ package anchor import ( + "bytes" "fmt" "log/syslog" "sync" @@ -37,13 +38,18 @@ func OpenSyslog(tag string) (*SyslogLog, error) { return &SyslogLog{w: w, now: time.Now}, nil } -// Write emits one record line to syslog. A write error means the record was -// not accepted. +// Write emits one record as a single syslog message. recordLine produces a +// newline-terminated NDJSON line for file/git sinks; syslog frames each call as +// its own message, so the trailing newline is stripped — leaving it in can make +// some syslog backends split one record into multiple log entries. The record +// JSON is itself single-line (no interior newlines), so trimming the final byte +// is sufficient to keep one checkpoint == one syslog event. func (s *SyslogLog) Write(eventType string, payload []byte) error { line, err := recordLine(s.now(), eventType, payload) if err != nil { return err } + line = bytes.TrimRight(line, "\n") s.mu.Lock() defer s.mu.Unlock() if _, err := s.w.Write(line); err != nil { diff --git a/daemon/internal/checkpoint/checkpoint.go b/daemon/internal/checkpoint/checkpoint.go index f679d59b..f4f70143 100644 --- a/daemon/internal/checkpoint/checkpoint.go +++ b/daemon/internal/checkpoint/checkpoint.go @@ -93,9 +93,17 @@ func Sign(cp Checkpoint, signer Signer) (Signed, error) { // Verify checks s's signature against the PEM/SPKI Ed25519 public key. It // re-canonicalises the embedded Checkpoint body (the wrapper fields are never -// signed) and validates the detached signature. A malformed signature, wrong -// key type, or mismatch returns (false, err) — checkpoint verification failure -// is surfaced, never swallowed. +// signed) and validates the detached signature. +// +// The two failure modes are distinct, so callers can tell a malformed artifact +// from a genuine forgery: +// - (false, err): the signature, key, or canonical body could not even be +// evaluated (bad multibase prefix, wrong length, unparseable key, etc.). +// - (false, nil): everything parsed but the signature does not match the body +// (a real verification failure / forgery). +// +// Only (true, nil) means the checkpoint is authentic. Either failure must be +// treated as a hard rejection — never swallowed. func Verify(s Signed, publicKeyPEM string) (bool, error) { if len(s.Signature) < 2 { return false, errors.New("checkpoint signature too short") diff --git a/daemon/internal/verifycli/anchor.go b/daemon/internal/verifycli/anchor.go index 61eb7f8e..62afbb63 100644 --- a/daemon/internal/verifycli/anchor.go +++ b/daemon/internal/verifycli/anchor.go @@ -2,7 +2,6 @@ package verifycli import ( "fmt" - "sort" "github.com/agent-receipts/ar/daemon/internal/checkpoint" ) @@ -43,7 +42,13 @@ func verifyAgainstAnchor(anchorPath, chainID, pubPEM string, headSeq int64, head return anchorResult{Reason: fmt.Sprintf("no verified checkpoint found for chain %s in anchor %s", chainID, anchorPath)} } - sort.Slice(cps, func(i, j int) bool { return cps[i].Sequence < cps[j].Sequence }) + // Validate strict monotonicity in ANCHOR FILE ORDER, never after a re-sort. + // The daemon appends a chain's checkpoints in strictly increasing sequence by + // construction (the emitter only writes a head past the highest it already + // anchored), so an out-of-order or duplicate record in the file is itself the + // tamper/corruption signal. Sorting first would launder "seq 2 then seq 1" + // into "1, 2" and pass — so the last record in file order is also the genuine + // latest head, not a max() that hides reordering. for i := 1; i < len(cps); i++ { if cps[i].Sequence <= cps[i-1].Sequence { return anchorResult{ diff --git a/daemon/internal/verifycli/anchor_test.go b/daemon/internal/verifycli/anchor_test.go index ca21ad20..8d2d8c17 100644 --- a/daemon/internal/verifycli/anchor_test.go +++ b/daemon/internal/verifycli/anchor_test.go @@ -173,3 +173,19 @@ func TestAnchorRejectsNonMonotonicLog(t *testing.T) { t.Errorf("reason %q does not flag the ordering problem", got.Reason) } } + +func TestAnchorRejectsOutOfOrderLog(t *testing.T) { + signer, pub := newAnchorSigner(t) + // seq 2 written before seq 1: the monotonic check runs in anchor FILE ORDER, + // so this must fail. Sorting first would launder it into 1,2 and pass, hiding + // a reordered/tampered log — guards against re-introducing a sort. + path := writeAnchor(t, signer, cp("c", 2, "sha256:2"), cp("c", 1, "sha256:1")) + + got := verifyAgainstAnchor(path, "c", pub, 2, "sha256:2", true) + if got.OK { + t.Fatal("expected FAIL for an out-of-order checkpoint log") + } + if !strings.Contains(got.Reason, "increasing") { + t.Errorf("reason %q does not flag the ordering problem", got.Reason) + } +} From e02a75a8d90ef7b58f6cbe5cb2b7de5a2d31257c Mon Sep 17 00:00:00 2001 From: Otto Jongerius Date: Tue, 16 Jun 2026 23:49:57 +0000 Subject: [PATCH 4/4] daemon: changelog entry for checkpoint anchor (v0.27.0-alpha.1) --- daemon/CHANGELOG.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/daemon/CHANGELOG.md b/daemon/CHANGELOG.md index 79a91c84..4275792b 100644 --- a/daemon/CHANGELOG.md +++ b/daemon/CHANGELOG.md @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.27.0-alpha.1] - 2026-06-16 + +### Added + +- **Out-of-band checkpoint anchor for tail-truncation resistance (ADR-0008 §2, spike #600).** The daemon can emit an additive, Ed25519-signed checkpoint of each chain HEAD — `{chain_id, sequence, receipt_hash, timestamp}`, canonicalised through the existing RFC 8785 path — to one or more append-only sinks the agent UID cannot rewrite. Enable with `--checkpoint-anchor`, a comma-separated fan-out list of `file:`, `git:`, or `syslog:` specs (env `AGENTRECEIPTS_CHECKPOINT_ANCHOR`, TOML `checkpoint_anchor`), and `--checkpoint-cadence` (receipts per checkpoint; default every receipt). A checkpoint is signed once and fanned out to every sink; the git sink commits each record so its commit chain is a tamper-evident log. Sink write failures are logged and metered but never block or undo receipt emission — receipts are the primary record. `obsigna receipt verify --against-anchor ` additionally verifies each checkpoint signature, asserts the log is strictly increasing in file order, and fails on tail truncation (a checkpoint ahead of the store HEAD). Receipts stay a linear verifiable-credential chain — this touches neither the receipt schema, hash chain, `@context`, nor the issuer DID (the anchoring freeze, ADR-0008). **Off by default:** with no `--checkpoint-anchor` configured, the daemon and `verify` are byte-identical to before. **Alpha — opt-in and experimental:** checkpoint emission is synchronous on the commit path, and durability/retry plus production-grade sinks (object-lock storage, TPM, transparency log) are not yet built. + ## [0.26.0] - 2026-06-14 ### Changed