Repository navigation
fix(docs): name v0.25.0 in README and gate release-claim drift - #657
toasterbook88 wants to merge 5 commits into
Conversation
v0.25.0 was tagged on 2026-11-16 but README.md still pinned v0.24.0 in the
three install commands and in the Status table, and SECURITY.md still
declared v0.24.x the supported line. Every gate passed: verify-doc-facts
proves each tag has a CHANGELOG entry, and verify-repo-truth rejects
unpublished tags and stale "current release is" claims, but nothing
compared the install pins or the Status table against the tags present in
the checkout. An operator following Quick start installed a superseded
binary, and the release flow never surfaced it.
- README: pin the three current-version commands to v0.25.0, point the
upgrade note at a v0.25.0 guide and push v0.24.0 into the older-release
list, and correct the Status table version, tag commit and supported
security line.
- SECURITY.md: supported line v0.25.x.
- docs/release-notes.md: add the v0.25.0 upgrade guide, derived from the
merged CHANGELOG entry (cloud stream idle guard, consult_model, desk
leases failing closed, A2A terminal-state refusal, daemon supervision
check, durable axis update).
- hack/verify-release-claims.sh: new local, network-free guard that the
first AXIS_VERSION pin, the Latest-release row (version and tag commit),
and the supported security line all name the newest release tag.
Historical material (rollback example, older-release links) stays free,
and a prerelease tag never demands promotion. Annotated tags are
dereferenced with ^{commit} because README quotes the commit, not the
tag object.
- hack/verify-release-claims-tests.sh: fixture-driven regression tests for
the happy path, each stale claim, the prerelease and no-tag cases, and
the missing-claim shapes. Wired into verify-doc-facts.sh, which CI, the
release workflow, and ci-preflight already run.
toasterbook88
left a comment
There was a problem hiding this comment.
AXIS Grok gate
Verdict: PASS-WITH-RISK
PR #657 — fix(docs): name v0.25.0 in README and gate release-claim drift
Head: 492856c85bb1fa996e1347f1566d2b9e06652ce2
Files: README.md, SECURITY.md, docs/release-notes.md, hack/verify-doc-facts.sh, hack/verify-release-claims.sh (new), hack/verify-release-claims-tests.sh (new)
Invariant hits: none.
- Docs + local network-free verification script only. No binary behavior change, no advisory surface, no HITL/confirm path, no cache badge change, no secret/ssh/fleet/spawn/self-authorize surface.
- New guard uses checkout tags as authority (grep +
git tag/rev-parse --short ^{commit}), scope-limited to the three “install this now” claims, prereleases ignored, historical material free. Wired into existingverify-doc-facts.shpath. - Aligns with AGENTS.md §2 (releases via tag workflow; never claim version from memory) and public-boundary rules.
CI status: in progress (Test & Build, govulncheck, Analyze). auto-merge skipped. No completed failures at review time.
Recommended operator action: Wait for CI green, then merge if the release-claim gate matches the intended freshness contract. No code changes required on this SHA.
Review event: COMMENT only. No APPROVE.
| # ^{commit} dereferences an annotated tag object to the commit it points at; | ||
| # README's "tag commit" quotes the commit (e.g. v0.24.0 -> 16eeb3b), never | ||
| # the tag object's own SHA. | ||
| latest_commit="$(git rev-parse --short "${latest}^{commit}" 2>/dev/null || true)" |
There was a problem hiding this comment.
🟡 Correct release commit fails validation
With Git's default abbreviation, latest_commit becomes 8578cb2, rejecting the correct README claim 8578cb2a. Consequently, verify-doc-facts.sh fails on this PR without overrides.
| latest_commit="$(git rev-parse --short "${latest}^{commit}" 2>/dev/null || true)" | |
| latest_commit="$(git rev-parse "${latest}^{commit}" 2>/dev/null || true)" |
Was this helpful? React with 👍 or 👎 to provide feedback.
| # and SECURITY.md still declaring v0.24.x supported, and every gate passed. | ||
| # The guard is separate so its regression tests can drive it with fixtures. | ||
| ./hack/verify-release-claims-tests.sh | ||
| ./hack/verify-release-claims.sh |
There was a problem hiding this comment.
🟡 Release publication blocked by freshness gate
When a new release tag is pushed, verify-release-claims.sh rejects the previous release retained by pre-tag validation. The publication job requires this gate, so publication cannot proceed.
Learn more
The release Test Gate runs this check before publication, and the publication job depends on that gate. Before tagging, the new check requires the README to name the previous release. After tagging the same commit, the check instead requires the new release. Updating the README before tagging fails pre-tag validation; updating it afterward cannot repair the tagged tree. Requiring the new tag's commit inside that same commit also creates a self-reference.
Example: With v0.25.0 available, a v0.25.1 preparation commit passes while README names v0.25.0. Pushing v0.25.1 on that commit makes the Test Gate reject v0.25.0, preventing GoReleaser from publishing.
Recommended fix: Make freshness validation release-phase-aware. During a tag build, validate operator claims against the preceding published release, or move the new-release freshness requirement to a post-publication check and documentation update. Add a regression covering pre-tag validation and tag-triggered publication of the same tree.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
The branch has been rebased onto
The PR remains clean and ready to merge. 📋 Walkthrough (6 files)
Automatic re-reviews are paused on this PR after 3 reviews. Mention @testdriverai to get a fresh review of the latest commits. Was this review useful? React with 👍 or 👎 to help TestDriver improve. Want help from the team that built TestDriver? Explore TestDriver for Business |
…gate The first CI run of the new gate failed while the same check passed locally: `git rev-parse --short` returns seven characters in a small CI clone and eight in a developer's repository, so comparing README's eight-character `tag commit` against a short reference rejected a correct citation. The reference is now the full 40-character commit, README may cite any 7-to-40-character prefix of it, and a reference that is not a full SHA fails with its own message instead of rejecting every citation. Verified in a shallow clone where `git rev-parse --short` reports seven characters (the CI shape): the gate passes and reports the deterministic eight-character form. Regression tests cover an eight-character citation against a full reference, the seven-character form, a citation shorter than seven characters, and a truncated reference.
`.github/workflows/release.yml` runs verify-doc-facts.sh in its `test` job with `fetch-depth: 0`, so on a tag push the new tag is already in the checkout while the tree still documents the previous release. Requiring README to name the newest tag failed that job on every release, and the README cannot cite the tag's own commit, which does not exist until the commit is made. Verified by tagging v0.25.1 on the current tree: the gate returned 1, which would have blocked GoReleaser. README may now name the newest release tag or the one immediately before it, and the cited `tag commit` is checked against that claimed tag rather than against the newest one. One release of lag is allowed because the tag push and the follow-up docs change cannot be the same commit; two are not. The drift this PR fixes still fails: when v0.25.0 was tagged with v0.24.2 already published, README named v0.24.0, which is two behind. The supported security line now keys off the claimed release instead of the newest tag, which is what keeps the window open across a minor bump. The redundant freshness test on the Status row is gone: the row must equal the Quick start pin, which the first check already compared. Regression tests pass an explicit tag/commit map, so the suite no longer reads the repository's own tags and cannot fail in a tagless checkout.
|
⏸️ Skipped this review: 3/3 PR reviews used this hour for this organization. Push a new commit after Tue, 17 Nov 2026 23:50:02 GMT and I'll review it. |
|
Both findings from the first review round are fixed. Recording the resolution so the threads are answered: 1. 2. 3. The release path (Devin, Reproduced by tagging Re-verified after the fix, in a clone whose
Both reviews above are anchored to |
Problem
v0.25.0was tagged on 2026-11-16, butREADME.mdstill pinned v0.24.0 in all three current-version install commands and in the Status table, andSECURITY.mdstill declared v0.24.x the supported line. The release notes had no v0.25.0 upgrade guide, andv0.24.2was already published at the time — so the docs were two releases behind.Every gate was green.
hack/verify-doc-facts.sh§5 proves each released tag has aCHANGELOG.mdentry, andhack/verify-repo-truth.shrejects unpublished tags and stalecurrent release isclaims — but nothing compared the install pins or the Status table against the tags present in the checkout. An operator following Quick start installed a superseded binary, and the release flow never surfaced it.Change
Fix the drift
README.md: pin the three current-version commands tov0.25.0; point the upgrade note at a v0.25.0 guide and movev0.24.0into the older-release list; correct the Status table version, tag commit (16eeb3b→8578cb2a) and supported security line.SECURITY.md: supported linev0.25.x.docs/release-notes.md: add the v0.25.0 upgrade guide, derived from the merged CHANGELOG entry — cloud stream idle guard,consult_model, desk leases failing closed, A2A terminal-state refusal, daemon supervision check, durableaxis update.Close the gap so it cannot recur
hack/verify-release-claims.sh— new local, network-free guard. The tags in the checkout are the authority. It checks the three claims that present themselves as install this now: the firstAXIS_VERSION=vX.Y.ZinREADME.md; the Status tableLatest releaserow (version andtag commit); and the supported security line, whichREADME.mdandSECURITY.mdmust agree on.hack/verify-release-claims-tests.sh— fixture-driven regression tests. Wired intoverify-doc-facts.sh, whichci.yml,release.yml, andhack/ci-preflight.shalready run.Two deliberate scope limits:
README.mdmay name the newest release tag or the one immediately before it.release.ymlruns this gate in itstestjob withfetch-depth: 0, so on a tag push the new tag is in the checkout while the tree still documents the previous release — and the README cannot cite the tag's own commit, which does not exist until the commit is made. One release of lag keeps publishing possible; two do not. The drift this PR fixes is still caught, becausev0.24.0was two behindv0.25.0.v0.26.0-rc.1) never demands promotion.Review round
Both automated reviewers flagged the same
--shortdefect on the first head, and Devin flagged a second defect that would have blocked publishing. Both were checked in code and reproduced.git rev-parse --shortyields 7 chars in CI and 8 locally, so the commit comparison rejected a correct citation (TestDriver, Devin)363f0dc0: the reference is the full 40-character commit,README.mdmay cite any 7-to-40-character prefix, and a truncated reference now fails with its own message instead of rejecting every citationrelease.ymltestjob runs this gate on a tag push, so the new tag would be rejected and GoReleaser would never run (Devin)v0.25.1on the tree: the gate returned 1d41db1f2: the two-tag window above, with the cited commit validated against the claimed tagd41db1f2: the suite passes an explicit tag/commit map, so it never reads the repo's own tags and cannot fail in a tagless checkoutReviewers' heads: both reviewed
492856c8. The head has moved twice since.Verification
verify-release-claims: README.md Quick start pins v0.24.0, but the tags say the newest release (v0.25.0) or the one before it (v0.24.2). It passes after the fix.v0.25.1tagged on this tree the gate passes and reportsnewest tag v0.25.1; with onlyv0.24.0–v0.25.0present and the README reverted tov0.24.0, it fails.git rev-parse --shortreturns 7 characters in that clone (the CI shape) and the gate passes — the earlier CI failure does not recur.hack/ci-preflight.sh— green end to end on363f0dc0: build,make test,make test-race,make coverage(total 78.8%),verify-public-boundary,verify-deadcode,verify-import-tiers,verify-specs,verify-repo-truth,verify-doc-facts, actionlint, lifecycle-check.Notes
mainred, since the gate correctly fails on the current tree.