Skip to content

Commit c2e2f51

Browse files
committed
docs(conformance): address code-review findings on the interop hardening
Follow-up to the IETF-audit hardening pass, fixing 8 review findings: - Page copy no longer claims counts "cannot silently drift" absolutely: the reconciliation test runs in cross-sdk-tests.yml, which triggers on the vector corpus / spec (not site/** or scripts/**), so the copy now states what is actually enforced (a corpus edit that outdates a number turns CI red). Test header comment records the same trigger-scope caveat. - Retract the "independent implementations" overclaim in the two docs the page links to (README.md, cross-sdk-tests/README.md); state the real property (separate, no-shared-code, single author; serialisation-drift detector, not spec-ambiguity detector). - ADR-0015: the dated 2026-05-12 snapshot's "Phase B ... deferred" line now forward-points to the update where Phase B checkpoint anchoring landed, so it no longer contradicts the header. - Reconciliation test: scope the matrix regex to the "## Results matrix" section and the prose regexes to the honesty callout, so an unrelated table or stray figure elsewhere on the page can't be misread; surface count.py's stderr on failure; load count.py output + page once and share across subtests (was exec'd/read twice). - Remove the unpinned "2-3 pinned receipt/chain vectors per spec version" prose figure, pointing to the (pinned) matrix instead. - Pin the verifier-side checkpoint reference to "ADR-0008 §3 out-of-band check over ADR-0015 Phase B checkpoints" (was bare "ADR-0008 follow-through").
1 parent 2f23604 commit c2e2f51

6 files changed

Lines changed: 145 additions & 103 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@
4949

5050
## Conformance
5151

52-
Anyone can implement the protocol — so interoperability is the property that matters. The three SDKs (Go, Python, TypeScript) are independent implementations that **verify one another's receipts**: every conformance vector is signed once with a shared Ed25519 keypair, and each SDK re-verifies the signatures, canonical JSON, and hashes the other two produced — across a positive corpus, a MUST-reject corpus, and four pinned spec versions. This cross-language verification is enforced in CI.
52+
Anyone can implement the protocol — so interoperability is the property that matters. The three SDKs (Go, Python, TypeScript) are separate, no-shared-code implementations that **verify one another's receipts**: every conformance vector is signed once with a shared Ed25519 keypair, and each SDK re-verifies the signatures, canonical JSON, and hashes the other two produced — across a positive corpus, a MUST-reject corpus, and four pinned spec versions. This cross-language verification is enforced in CI (it catches serialisation drift between the languages, not spec-interpretation ambiguity — the SDKs share one author).
5353

5454
The **[Conformance page](https://agentreceipts.ai/conformance/)** is the citeable summary: the interop claim, a results matrix with per-set vector counts, permanent links to every frozen vector set, and how to reproduce the checks. The shared corpus lives in [`cross-sdk-tests/`](cross-sdk-tests/).
5555

cross-sdk-tests/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Cross-SDK Tests
22

3-
Three independent implementations of the Agent Receipt Protocol — the Go, Python, and TypeScript SDKs — verify one another's receipts. The vectors in this directory are the shared corpus that makes that claim checkable: each is signed once with a single shared Ed25519 keypair, and every SDK re-verifies the signatures, canonical JSON (RFC 8785), and receipt hashes the other two produced. Verification runs in both directions, over a **positive corpus** every implementation must accept and a **MUST-reject corpus** every implementation must refuse, across four pinned spec versions (v0.2.0–v0.5.0).
3+
Three separate implementations of the Agent Receipt Protocol — the Go, Python, and TypeScript SDKs, one per language with no shared code — verify one another's receipts. The vectors in this directory are the shared corpus that makes that claim checkable: each is signed once with a single shared Ed25519 keypair, and every SDK re-verifies the signatures, canonical JSON (RFC 8785), and receipt hashes the other two produced. Verification runs in both directions, over a **positive corpus** every implementation must accept and a **MUST-reject corpus** every implementation must refuse, across four pinned spec versions (v0.2.0–v0.5.0). This catches cross-language serialisation drift; because the SDKs share a single author, it does not catch a spec-interpretation ambiguity encoded identically into all three (see the [Conformance page](https://agentreceipts.ai/conformance/) for the full statement of what this does and does not prove).
44

55
For the published, citeable summary of this suite — the interop claim, the results matrix with per-set vector counts, and CI-enforcement status — see the [Conformance page](https://agentreceipts.ai/conformance/).
66

cross-sdk-tests/conformance_page_test.go

Lines changed: 138 additions & 97 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,8 @@ package crosssdk_test
1010
// is the generator — it reads the frozen vector files and emits the counts.
1111
// Nothing, though, forced the page's numbers to equal count.py's output: the
1212
// table cells and the prose figures were hand-typed, so a change to a vector
13-
// file (or a typo in the page) could silently desync the two. On a page that is
14-
// cited to a standards body as evidence, a stale number is worse than none.
13+
// file could silently desync the two. On a page that is cited to a standards
14+
// body as evidence, a stale number is worse than none.
1515
//
1616
// This test closes that gap. It runs count.py, parses the page, and asserts:
1717
// - every on-page vector set from count.py appears as a matrix row with the
@@ -21,11 +21,16 @@ package crosssdk_test
2121
// total and the MUST-reject case count) match count.py.
2222
//
2323
// It lives here, in cross-sdk-tests/, because cross-sdk-tests.yml re-runs on
24-
// every change to the vector corpora (cross-sdk-tests/** and spec/**) — exactly
25-
// the changes that move these counts. count.py is invoked as the single source
26-
// of truth rather than re-deriving counts here, so the two cannot drift.
24+
// every change to the vector corpora (cross-sdk-tests/** and spec/**) — the
25+
// changes that actually move these counts. That path filter does NOT cover
26+
// site/** or scripts/**, so a page-only edit (or a count.py-only edit) that is
27+
// not accompanied by a corpus change is outside this gate's CI trigger;
28+
// widening the trigger to those paths is a separate workflow change. count.py
29+
// is invoked as the single source of truth rather than re-deriving counts here,
30+
// so the two cannot drift.
2731

2832
import (
33+
"bytes"
2934
"encoding/json"
3035
"os"
3136
"os/exec"
@@ -51,19 +56,32 @@ type countPayload struct {
5156
Total int `json:"total"`
5257
}
5358

54-
// runCountScript executes count.py --format json and returns the parsed payload.
55-
// If python3 is not on PATH (e.g. a bare local `go test` without Python), the
56-
// test is skipped rather than failed — CI runners have python3, so the gate
57-
// still fires where it matters.
58-
func runCountScript(t *testing.T) countPayload {
59+
// reconcileInputs bundles the once-loaded count.py output and page text so the
60+
// subtests share a single python3 invocation and a single page read.
61+
type reconcileInputs struct {
62+
payload countPayload
63+
counts map[string]int // set name -> count, for O(1) lookup
64+
page string
65+
}
66+
67+
// loadReconcileInputs runs count.py --format json and reads the page. If python3
68+
// is not on PATH (e.g. a bare local `go test` without Python), the test is
69+
// skipped rather than failed — CI runners have python3, so the gate still fires
70+
// where it matters.
71+
func loadReconcileInputs(t *testing.T) reconcileInputs {
5972
t.Helper()
6073
python, err := exec.LookPath("python3")
6174
if err != nil {
6275
t.Skipf("python3 not available, skipping conformance-page reconciliation: %v", err)
6376
}
64-
out, err := exec.Command(python, countScriptPath, "--format", "json").Output()
77+
cmd := exec.Command(python, countScriptPath, "--format", "json")
78+
var stderr bytes.Buffer
79+
cmd.Stderr = &stderr
80+
out, err := cmd.Output()
6581
if err != nil {
66-
t.Fatalf("run count.py: %v", err)
82+
// count.py prints "error: could not count vectors: <detail>" to stderr;
83+
// surface it so a broken vector file is diagnosable from the CI log.
84+
t.Fatalf("run count.py: %v\nstderr: %s", err, strings.TrimSpace(stderr.String()))
6785
}
6886
var payload countPayload
6987
if err := json.Unmarshal(out, &payload); err != nil {
@@ -72,19 +90,58 @@ func runCountScript(t *testing.T) countPayload {
7290
if len(payload.Sets) == 0 {
7391
t.Fatal("count.py returned no vector sets")
7492
}
75-
return payload
93+
counts := make(map[string]int, len(payload.Sets))
94+
for _, s := range payload.Sets {
95+
counts[s.Name] = s.Count
96+
}
97+
pageBytes, err := os.ReadFile(conformancePage)
98+
if err != nil {
99+
t.Fatalf("read conformance page: %v", err)
100+
}
101+
return reconcileInputs{payload: payload, counts: counts, page: string(pageBytes)}
102+
}
103+
104+
// section returns the slice of the page between the first line starting with
105+
// startPrefix (exclusive) and the next line starting with endPrefix (exclusive).
106+
// Scoping the regex scans to the intended block means an unrelated table or a
107+
// stray prose figure elsewhere on the page cannot be misread as a matrix row or
108+
// as a pinned count.
109+
func section(t *testing.T, page, startPrefix, endPrefix string) string {
110+
t.Helper()
111+
lines := strings.Split(page, "\n")
112+
start := -1
113+
for i, line := range lines {
114+
if strings.HasPrefix(line, startPrefix) {
115+
start = i + 1
116+
break
117+
}
118+
}
119+
if start == -1 {
120+
t.Fatalf("section start %q not found on the page", startPrefix)
121+
}
122+
end := len(lines)
123+
for i := start; i < len(lines); i++ {
124+
if strings.HasPrefix(lines[i], endPrefix) {
125+
end = i
126+
break
127+
}
128+
}
129+
if end <= start {
130+
t.Fatalf("section %q..%q is empty", startPrefix, endPrefix)
131+
}
132+
return strings.Join(lines[start:end], "\n")
76133
}
77134

78135
// matrixRow matches a results-matrix data row: a table row whose first cell is a
79136
// markdown link, e.g. `| [canonicalization](https://…) | … | 44 |`. The link
80137
// text is the vector-set name; the final numeric cell is the count.
81138
var matrixRowRe = regexp.MustCompile(`^\|\s*\[([^\]]+)\]\([^)]*\)\s*\|.*\|\s*([0-9]+)\s*\|\s*$`)
82139

83-
// parsePageMatrix extracts {name: count} for every results-matrix row on the page.
84-
func parsePageMatrix(t *testing.T, page string) map[string]int {
140+
// parseMatrix extracts {name: count} for every matrix row in the given section.
141+
func parseMatrix(t *testing.T, src string) map[string]int {
85142
t.Helper()
86143
rows := map[string]int{}
87-
for _, line := range strings.Split(page, "\n") {
144+
for _, line := range strings.Split(src, "\n") {
88145
m := matrixRowRe.FindStringSubmatch(line)
89146
if m == nil {
90147
continue
@@ -99,97 +156,81 @@ func parsePageMatrix(t *testing.T, page string) map[string]int {
99156
rows[m[1]] = n
100157
}
101158
if len(rows) == 0 {
102-
t.Fatal("no matrix rows parsed from the conformance page")
159+
t.Fatal("no matrix rows parsed from the results-matrix section")
103160
}
104161
return rows
105162
}
106163

107-
func TestConformancePageMatrixReconciles(t *testing.T) {
108-
payload := runCountScript(t)
109-
pageBytes, err := os.ReadFile(conformancePage)
110-
if err != nil {
111-
t.Fatalf("read conformance page: %v", err)
112-
}
113-
page := string(pageBytes)
114-
pageRows := parsePageMatrix(t, page)
115-
116-
// Every on-page set from count.py must appear with the exact computed count.
117-
expected := map[string]int{}
118-
for _, s := range payload.Sets {
119-
if !s.OnPage {
120-
// Off-page sets (reference fixtures) must NOT appear as matrix rows.
121-
if _, present := pageRows[s.Name]; present {
122-
t.Errorf("set %q is on_page=False in count.py but appears in the page matrix", s.Name)
123-
}
124-
continue
125-
}
126-
expected[s.Name] = s.Count
127-
got, present := pageRows[s.Name]
128-
if !present {
129-
t.Errorf("count.py set %q is missing from the page matrix", s.Name)
130-
continue
131-
}
132-
if got != s.Count {
133-
t.Errorf("count mismatch for %q: page=%d, count.py=%d", s.Name, got, s.Count)
134-
}
135-
}
136-
// No matrix row may exist that count.py does not account for.
137-
for name := range pageRows {
138-
if _, ok := expected[name]; !ok {
139-
t.Errorf("page matrix row %q has no corresponding on-page set in count.py", name)
140-
}
141-
}
142-
}
143-
144-
// setCount returns the count.py count for a named set.
145-
func setCount(t *testing.T, payload countPayload, name string) int {
146-
t.Helper()
147-
for _, s := range payload.Sets {
148-
if s.Name == name {
149-
return s.Count
150-
}
151-
}
152-
t.Fatalf("count.py has no set named %q", name)
153-
return 0
154-
}
155-
156164
var (
157165
proseCanonRe = regexp.MustCompile(`(\d+) canonicalization/receipt-hash vectors`)
158166
proseMalformedRe = regexp.MustCompile(`(\d+)-case MUST-reject corpus`)
159167
)
160168

161-
// TestConformancePageProseFiguresReconcile pins the two exact numeric claims in
162-
// the honesty callout to count.py, so the prose cannot drift from the matrix
163-
// either.
164-
func TestConformancePageProseFiguresReconcile(t *testing.T) {
165-
payload := runCountScript(t)
166-
pageBytes, err := os.ReadFile(conformancePage)
167-
if err != nil {
168-
t.Fatalf("read conformance page: %v", err)
169-
}
170-
page := string(pageBytes)
171-
172-
checks := []struct {
173-
label string
174-
re *regexp.Regexp
175-
setName string
176-
}{
177-
{"canonicalization total", proseCanonRe, "canonicalization"},
178-
{"MUST-reject case count", proseMalformedRe, "malformed (MUST-reject)"},
179-
}
180-
for _, c := range checks {
181-
m := c.re.FindStringSubmatch(page)
182-
if m == nil {
183-
t.Errorf("%s: prose figure not found on the page (pattern %q)", c.label, c.re)
184-
continue
169+
func TestConformancePageReconciles(t *testing.T) {
170+
in := loadReconcileInputs(t)
171+
172+
// The published matrix lives under "## Results matrix" and ends at the
173+
// "Each linked vector set…" sentence right after the table.
174+
t.Run("matrix", func(t *testing.T) {
175+
matrixSrc := section(t, in.page, "## Results matrix", "Each linked vector set")
176+
pageRows := parseMatrix(t, matrixSrc)
177+
178+
expected := map[string]int{}
179+
for _, s := range in.payload.Sets {
180+
if !s.OnPage {
181+
// Off-page sets (reference fixtures) must NOT appear as matrix rows.
182+
if _, present := pageRows[s.Name]; present {
183+
t.Errorf("set %q is on_page=False in count.py but appears in the page matrix", s.Name)
184+
}
185+
continue
186+
}
187+
expected[s.Name] = s.Count
188+
got, present := pageRows[s.Name]
189+
if !present {
190+
t.Errorf("count.py set %q is missing from the page matrix", s.Name)
191+
continue
192+
}
193+
if got != s.Count {
194+
t.Errorf("count mismatch for %q: page=%d, count.py=%d", s.Name, got, s.Count)
195+
}
185196
}
186-
got, err := strconv.Atoi(m[1])
187-
if err != nil {
188-
t.Fatalf("%s: non-numeric prose figure %q", c.label, m[1])
197+
// No matrix row may exist that count.py does not account for.
198+
for name := range pageRows {
199+
if _, ok := expected[name]; !ok {
200+
t.Errorf("page matrix row %q has no corresponding on-page set in count.py", name)
201+
}
189202
}
190-
want := setCount(t, payload, c.setName)
191-
if got != want {
192-
t.Errorf("%s: prose says %d, count.py says %d", c.label, got, want)
203+
})
204+
205+
// The two exact numeric claims in the honesty callout are pinned to count.py
206+
// too, scoped to that Aside so an earlier figure elsewhere cannot shadow them.
207+
t.Run("prose", func(t *testing.T) {
208+
callout := section(t, in.page, `<Aside type="caution" title="Honest coverage`, "</Aside>")
209+
checks := []struct {
210+
label string
211+
re *regexp.Regexp
212+
setName string
213+
}{
214+
{"canonicalization total", proseCanonRe, "canonicalization"},
215+
{"MUST-reject case count", proseMalformedRe, "malformed (MUST-reject)"},
193216
}
194-
}
217+
for _, c := range checks {
218+
m := c.re.FindStringSubmatch(callout)
219+
if m == nil {
220+
t.Errorf("%s: prose figure not found in the honesty callout (pattern %q)", c.label, c.re)
221+
continue
222+
}
223+
got, err := strconv.Atoi(m[1])
224+
if err != nil {
225+
t.Fatalf("%s: non-numeric prose figure %q", c.label, m[1])
226+
}
227+
want, ok := in.counts[c.setName]
228+
if !ok {
229+
t.Fatalf("count.py has no set named %q", c.setName)
230+
}
231+
if got != want {
232+
t.Errorf("%s: prose says %d, count.py says %d", c.label, got, want)
233+
}
234+
}
235+
})
195236
}

daemon/internal/verifycli/anchor.go

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,8 @@ type anchorResult struct {
1818
}
1919

2020
// verifyAgainstAnchor checks the receipt chain HEAD against the out-of-band
21-
// signed checkpoints in the anchor at anchorPath (ADR-0008 follow-through).
21+
// signed checkpoints in the anchor at anchorPath (the ADR-0008 §3 out-of-band
22+
// check over ADR-0015 Phase B checkpoints).
2223
//
2324
// It performs, in order:
2425
// - read the anchor log and retrieve the latest verified checkpoint for

docs/adr/0015-key-rotation-byok-anchoring.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,7 @@ What has **not** landed and is in scope for Phase A:
138138
- `Rotate()` currently returns `ErrNotImplemented`. No rotation event schema, no `key_rotated` synthetic receipt emission path, no verifier-side rotation traversal.
139139
- The external anchor write contract for rotation events. Until the anchor sink exists, **the post-compromise integrity guarantee is not honoured** and any rotation work that lands before it is partial.
140140

141-
Phase B (checkpoint anchoring) and Phase C (HSM/KMS adapters) are explicitly deferred and not on a schedule.
141+
Phase B (checkpoint anchoring) and Phase C (HSM/KMS adapters) are explicitly deferred and not on a schedule *at this snapshot* — see the updates below, where Phase B checkpoint anchoring has since landed end-to-end. Phase C remains deferred.
142142

143143
### Update (2026-06-10): rotation verification and offline rotation landed
144144

site/src/content/docs/conformance.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ The three SDKs share no code: each is a separate implementation of the same spec
2323

2424
## Results matrix
2525

26-
Each row is a frozen vector set. The **Go / Py / TS** columns mark which implementations consume and verify that set in their test suites. Vector counts are generated from the committed JSON by [`scripts/conformance_matrix/count.py`](https://github.com/agent-receipts/obsigna/blob/main/scripts/conformance_matrix/count.py), and a CI [reconciliation test](https://github.com/agent-receipts/obsigna/blob/main/cross-sdk-tests/conformance_page_test.go) asserts every count in this table equals that script's outputso a number here cannot silently drift from the corpus.
26+
Each row is a frozen vector set. The **Go / Py / TS** columns mark which implementations consume and verify that set in their test suites. Vector counts are generated from the committed JSON by [`scripts/conformance_matrix/count.py`](https://github.com/agent-receipts/obsigna/blob/main/scripts/conformance_matrix/count.py), and a CI [reconciliation test](https://github.com/agent-receipts/obsigna/blob/main/cross-sdk-tests/conformance_page_test.go) asserts every count in this table — and the figures in the callout below — equal that script's output. It re-runs whenever the vector corpus or the spec changes, so a corpus edit that outdates a number here turns the check red before merge.
2727

2828
| Vector set | Purpose | Go | Py | TS | Spec version(s) | Vectors |
2929
| --- | --- | :-: | :-: | :-: | --- | ---: |
@@ -40,7 +40,7 @@ Each row is a frozen vector set. The **Go / Py / TS** columns mark which impleme
4040
Each linked vector set above is pinned to an immutable commit, so a citation stays valid as the corpus grows.
4141

4242
<Aside type="caution" title="Honest coverage — these are seed-level counts">
43-
The counts are the real sizes of the frozen corpora today, not a coverage claim. They are seed-level and growing: 44 canonicalization/receipt-hash vectors, a 7-case MUST-reject corpus, and 2–3 pinned receipt/chain vectors per spec version. They are **not** exhaustive — they pin the shapes that mattered enough to freeze, and the corpus is extended as the specification evolves. Separately, the `did:key` resolution vectors ([`spec/test-vectors/did-key/`](https://github.com/agent-receipts/obsigna/tree/main/spec/test-vectors/did-key)) fix the `did:key` v0.7 wire shape (ADR-0007) as reference fixtures, but **no SDK implements `did:key` resolution yet**, so no suite consumes them. Rather than show an all-dashes row, that set is out-of-scope-for-now and omitted from the matrix above; wiring it into a cross-language suite is tracked in [#956](https://github.com/agent-receipts/obsigna/issues/956).
43+
The counts are the real sizes of the frozen corpora today, not a coverage claim. They are seed-level and growing: 44 canonicalization/receipt-hash vectors, a 7-case MUST-reject corpus, and the per-version receipt/chain counts shown in the matrix above. They are **not** exhaustive — they pin the shapes that mattered enough to freeze, and the corpus is extended as the specification evolves. Separately, the `did:key` resolution vectors ([`spec/test-vectors/did-key/`](https://github.com/agent-receipts/obsigna/tree/main/spec/test-vectors/did-key)) fix the `did:key` v0.7 wire shape (ADR-0007) as reference fixtures, but **no SDK implements `did:key` resolution yet**, so no suite consumes them. Rather than show an all-dashes row, that set is out-of-scope-for-now and omitted from the matrix above; wiring it into a cross-language suite is tracked in [#956](https://github.com/agent-receipts/obsigna/issues/956).
4444
</Aside>
4545

4646
## CI enforcement

0 commit comments

Comments
 (0)