Skip to content

examples: bind MCP retries to retained declaration snapshots - #423

Merged
imran-siddique merged 7 commits into
agentrust-io:mainfrom
noah-ing:examples/mcp-retry-evidence
Oct 2, 2026
Merged

imran-siddique merged 7 commits into
agentrust-io:mainfrom
noah-ing:examples/mcp-retry-evidence

Conversation

@noah-ing

@noah-ing noah-ing commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds an executable worked example for the lost-response/retry/changed-declarations
case requested in #324. The session-wording work already landed in #413.

Run python examples/mcp-retry/mcp_retry.py --out DIR to write one signed TRACE
v0.2 record, its full transcript and declaration snapshots, and separate verification
inputs. The first attempt loses its response stream and stays unknown. Its retry
uses a new request ID and records success; an uncalled tool's declaration changes
between the two complete paginated captures.

The script is the single source; generated JSON is not committed. Tests run it in
two isolated directories, pin both outputs to their original byte hashes, verify
the record with a separately configured key, and recompute the full transcript
and snapshot commitments without the generator's helper. The test-only recomputation
checks the supported numeric domain before hash comparison and distinguishes
matched, mismatch, unavailable and unsupported. Both paths share rfc8785;
this is not independent validation of that canonicalizer.

No SDK, schema, dependency or acceptance-rule changes. The format and safe-integer
canonicalization are example-local, not an adopted v0.3 profile or live MCP
integration. Producer observations do not prove execution, authorization, server
identity, complete history, hardware provenance or exactly-once behavior. Remaining
design work in #324 stays outside scope.

Verification at df63ca4

  • Merged upstream main at 0014764, retaining every upstream changelog and
    examples-index entry alongside ours. GitHub reports no merge conflict.
    The example, its tests and the profile note are unchanged from approved 389459c.
  • 48 example tests and all 96 focused example/regeneration/corpus checks passed.
    The numeric-domain regression was established at 389459c: 11 new cases failed
    before the helper fix and passed afterward, including fractional input supplied
    with its matching raw-JCS digest.
  • Full host suite: 3,330 passed, 48 upstream conditional skips, no failures.
    Skips cover optional cross-repository dependencies, unavailable captured evidence
    or verifier snapshots, and existing environment/vector conditions. No example
    tests are skipped; this PR adds no skips or xfails.
  • Both generated output hashes remain unchanged. No producer-format changes.
  • Ruff, formatting, prose, mypy (14 source files) and whitespace checks passed.
  • Wheel and sdist builds passed. A separate merge audit confirmed upstream content
    was preserved and the previously approved implementation/tests were unchanged.

Current-head Python 3.11/3.12 CI
and CodeQL passed.
Each CI test job reports 3,330 passed and 48 conditional skips. The approval gate
requires a maintainer's approval of the new head. No local Windows or Docker
execution is claimed for this head. The earlier 58e8659 verification included
fresh-install package checks and a locked-dependency audit; those remain prior-head
results.

Signed-off-by: Noah Ingwers <98993329+noah-ing@users.noreply.github.com>
@noah-ing
noah-ing requested review from a team, lywinged and rajnisht7 as code owners September 26, 2026 12:32
@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

🟡 Contributor Check: MEDIUM

Check Result
Profile MEDIUM
Credential LOW
Overall MEDIUM

Automated check by AgenTrust Contributor Check.

@github-actions github-actions Bot added the needs-review:MEDIUM Contributor check flagged MEDIUM risk label Sep 26, 2026
Signed-off-by: Noah Ingwers <98993329+noah-ing@users.noreply.github.com>
Signed-off-by: Noah Ingwers <98993329+noah-ing@users.noreply.github.com>

@rajnisht7 rajnisht7 left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the head 58e8659, the changes looks good, the example is clearly scoped (local formats, v0.2 record, not a v0.3 profile), the lost-response / retry / retained-snapshot story matches what #324 was asking to illustrate, and the tests also cover the important integrity paths well, also would not treat this as closing #324 as the normative design work still sits elsewhere, but as an informative example it is in good shape.

a few nits:

  • _commitment is independent of the generator helper, but both sides still go through the same rfc8785 library. A shared mistake in that layer would not be caught here. Fine for an example though.

  • snapshots = {digest(s): s for s in (before, after)} would silently collapse if two snapshots ever hashed the same. The current fixture differs on purpose, so this is not a bug but an explicit len(snapshots) == 2 (or an ordered list of pairs) will make the invariant easier to see

  • Numeric refusal is tested via load_json / canonical_bytes, not via digest() itself. Today it is fine because digest() calls canonical_bytes(). A later change that had digest() call rfc8785.dumps directly could slip past those tests.

  • Attempt order currently rides on dict insertion order. With Python ≥ 3.11 that is deterministic and correct here; wiring attempt 1 -> before and attempt 2 -> after through an explicit list would just be a bit easier to audit.

  • Path.write_text() with default newlines can turn \n into \r\n on windows, so the exact-byte SHA pins may fail there even when the JSON is fine. Prefer write_bytes(...encode("utf-8")) or write_text(..., newline="\n")

Signed-off-by: Noah Ingwers <98993329+noah-ing@users.noreply.github.com>
@noah-ing

Copy link
Copy Markdown
Contributor Author

Thanks for the careful review. Addressed in d51f443:

  • Attempt order now comes from explicit before/after pairs, with an assertion that both snapshot digests survive in the lookup map; tests pin the two capture IDs to their attempts.
  • The numeric-refusal cases now call digest() directly as well as load_json(). A trial bypass of the canonical-value guard is caught by the new coverage.
  • Output uses write_bytes() with explicit UTF-8 encoding, avoiding Windows newline translation. Both existing output hashes are unchanged; I have not run Windows locally.
  • The README and test comment now explicitly state that both commitment paths share rfc8785, not an independently validated canonicalizer.

37 example tests and all 77 focused checks pass; the full host suite is 2,432 passed with six existing conditional skips. Ruff, formatting, prose, mypy and wheel/sdist build checks pass. The example remains informative; the remaining normative work in #324 is outside its scope.

rajnisht7
rajnisht7 previously approved these changes Sep 26, 2026

@rajnisht7 rajnisht7 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the head d51f443, the nits are addressed, portable UTF-8/LF writes via write_bytes, explicit capture ordering with a two-digest uniqueness assert, digest() covered in the numeric refusal tests, and the shared rfc8785 limit called out in the README and test comment. Still would not treat this as closing #324, but as an informative example this is in good shape.

@imran-siddique

Copy link
Copy Markdown
Member

@noah-ing this is approved and green; the only blocker is CHANGELOG.md, where main added entries at the top of [Unreleased]. Merge main in and keep main's entries with yours alongside them. #403 appends to the same block, so whichever of the two lands first, the other will need one more merge. Once you push it I will merge.

Copy link
Copy Markdown
Contributor

I ran one more focused pass on head d51f443 and found one residual verifier/producer asymmetry in the example-local canonicalization contract.

The example declares example-only/jcs-safe-integers-v1 and the producer enforces that domain in canonical_bytes(): all floats are refused, unsafe integers are refused, and digest() therefore has no valid commitment for e.g. {"x": 1.5}.

The verification helper does something different:

def _commitment(value, expected):
    if value is None:
        return "unavailable"
    actual = "sha256:" + hashlib.sha256(rfc8785.dumps(value)).hexdigest()
    return "matched" if actual == expected else "mismatch"

That path recomputes raw RFC 8785 bytes without independently enforcing the example's narrower supported-value domain. A finite fractional number is valid RFC 8785 input, so a value the declared example format says is unsupported can still be classified by the verification path as matched if expected is the corresponding JCS digest.

Concrete counterexample shape:

producer:
{"x": 1.5}
→ unsupported / no valid example-format commitment

verification helper:
{"x": 1.5}
+ expected = SHA256(JCS({"x":1.5}))
→ matched

This is not a signature bypass and does not make the current fixed packet malleable: changing a committed integer to a fraction still changes the digest. The issue is narrower — the verifier can positively match evidence that lies outside the canonicalization contract it says it is verifying.

The current numeric-refusal regression reaches load_json() and digest(), but not _commitment(), so it does not exercise this asymmetry.

A bounded fix would keep the recomputation independent of the generator helper, but give the verification path its own validation of the declared domain before calling rfc8785.dumps(), ideally distinguishing:

  • matched
  • mismatch
  • unavailable
  • unsupported

The decisive regression would be a float-containing retained object plus its otherwise matching RFC 8785 digest, which should report unsupported, never matched.

This seems worth fixing in the example because §3 of the draft already says unsupported canonical values must be reported rather than coerced/dropped, and the §6 acceptance table calls for unsupported numeric values to be rejected or reported unsupported.

Signed-off-by: Noah Ingwers <98993329+noah-ing@users.noreply.github.com>
Signed-off-by: Noah Ingwers <98993329+noah-ing@users.noreply.github.com>
@noah-ing

Copy link
Copy Markdown
Contributor Author

Thanks both. Merged main at 0d0e22b in d2608ca, preserving every upstream [Unreleased] entry with the MCP example entry alongside them. The branch is conflict-free again.

Agreed on the narrower domain mismatch: _commitment() was comparing raw JCS rather than enforcing the example's supported numeric domain. Fixed in 389459c with its own numeric check before hashing, without calling the generator helper. It now returns unsupported for floats and unsafe integers, separately from matched, mismatch and unavailable.

The 11 new cases failed before the fix and pass afterward, including 1.5 paired with its otherwise matching raw-JCS digest. Supported booleans, nested nulls and safe-integer boundaries still match. Both generated artifact hashes are unchanged.

48 example tests, all 88 focused checks, and the full host suite (2,443 passed, six existing conditional skips) pass. Lint, formatting, prose, mypy and wheel/sdist builds pass. Current-head Python 3.11/3.12 CI and CodeQL are green; the gate awaits approval of the new head. The fix stays in the test helper and its documentation; no SDK or normative behavior changed.

imran-siddique
imran-siddique previously approved these changes Sep 30, 2026

@imran-siddique imran-siddique left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@noah-ing approving 389459c; altrudev's numeric-domain point is fixed as described. #437 and #432 merged today and put CHANGELOG.md back in conflict, only that file. One more merge of main keeping every [Unreleased] entry and I'll merge it the same day.

Signed-off-by: Noah Ingwers <98993329+noah-ing@users.noreply.github.com>
@noah-ing

noah-ing commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Thanks. Merged current main (0014764) in df63ca4, preserving every upstream [Unreleased] entry alongside ours. Main had also added two entries to examples/README.md; those are retained too. The example, tests and profile note are unchanged from approved 389459c.

All 48 example tests and 96 focused checks pass. The full host suite passed with 3,330 tests and 48 upstream conditional skips; lint, formatting, prose, mypy and wheel/sdist builds passed. Both generated output hashes are unchanged. Python 3.11/3.12 CI and CodeQL passed. GitHub reports no merge conflict; only current-head maintainer approval remains.

@imran-siddique
imran-siddique merged commit 7d1289a into agentrust-io:main Oct 2, 2026
6 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review:MEDIUM Contributor check flagged MEDIUM risk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants