Repository navigation
feat(ops): add a versioned Vercel Preview retention runner - #1032
Conversation
Replace the scratchpad script behind the post-merge Vercel Preview reconciliation with a reviewed runner under scripts/vercel-retention. pnpm run vercel:retention:dry-run is read-only and pnpm run vercel:retention:reconcile is the full pass; both take no argument, so a command-line allow rule for them cannot be widened by appending text. A pass proves the exact-main gates (every CI / CD and CodeQL SAST run on the SHA, Production READY on it, the three canonical aliases, canonical HTTP) and the fixed project identity, reads every page of deployments and aliases, classifies each preview as SAFE, PROTECTED or UNKNOWN in a pure function, re-reads main, the gates and the provider before deleting each ID, removes the branch alias of a gone branch first, never retries a write and ends with a post-proof. A mutating pass runs only when the runner and the two package scripts equal the reviewed origin/main. Mutations validate one deployment ID or alias host and always pass --safe, so a project name cannot reach the project-wide form of remove. Tests run against a simulated provider and cover authority, pagination, project identity, Production protection, open-PR retention, unknown metadata, fault handling, idempotence, redaction and the argument-free entry points.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
🤖 CodeAnt AI — Review Status
|
Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
|
Warning Review limit reachedEnable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. Next included review available in 48 minutes. View limit details
📝 Walkthrough
Priority: ⬇️ Low Merge Risk: 🔵 Low · up to
|
Reviewer's GuideIntroduces a reviewed, versioned Vercel Preview retention runner that proves exact-main and production safety gates, inventories and classifies provider state conservatively, performs bounded per-ID revalidated deletion only from a trusted checkout, and verifies the resulting state with comprehensive simulated-provider tests. Sequence diagram for the Vercel retention reconciliation passsequenceDiagram
participant CLI
participant Runner
participant GitHub
participant Vercel
participant HTTP
CLI->>Runner: parseMode(dry-run or reconcile)
alt reconcile
Runner->>GitHub: assertTrustedSource()
end
Runner->>GitHub: readMain()
Runner->>Vercel: readProject()
Runner->>Vercel: listDeployments()
Runner->>Vercel: listAliases()
Runner->>GitHub: readOpenPrs()
Runner->>GitHub: readBranches()
Runner->>GitHub: assertRequiredRuns(main SHA)
Runner->>Vercel: assertProduction(world, main SHA)
Runner->>HTTP: GET canonical URL
Runner->>Runner: classify(world)
alt reconcile with SAFE candidates
loop Up to 25 IDs
Runner->>GitHub: readMain()
Runner->>Vercel: readProject(), listDeployments(), listAliases()
Runner->>GitHub: readOpenPrs(), readBranches()
Runner->>Runner: classify(world)
alt verdict remains SAFE
Runner->>Vercel: removeAlias(host)
Runner->>Vercel: removeDeployment(id)
Runner->>Vercel: listDeployments()
else verdict changed
Runner->>Runner: defer ID
end
end
Runner->>Vercel: listDeployments(), listAliases()
Runner->>HTTP: GET canonical URL
end
Runner-->>CLI: RETENTION_SUMMARY
State diagram for Vercel retention deployment verdictsstateDiagram-v2
[*] --> ProductionClass: isProductionClass
[*] --> Preview: deployment is not production-class
Preview --> PROTECTED: protectionReasons found
Preview --> UNKNOWN: unknownReasons or lifecycleUnknown found
Preview --> SAFE: no protection or unknown reasons
SAFE --> Revalidate: reconcile candidate
Revalidate --> SAFE: verdict unchanged
Revalidate --> Deferred: verdict changed
SAFE --> Deleted: removeDeployment confirmed
SAFE --> Stopped: write failed or unconfirmed
PROTECTED --> [*]
UNKNOWN --> [*]
Deleted --> [*]
Deferred --> [*]
Stopped --> [*]
Flow diagram for conservative Vercel preview classificationflowchart TD
A[Provider deployment] --> B{isProductionClass}
B -->|yes| P[PROTECTED production class]
B -->|no| C{identity and metadata valid}
C -->|no| U[UNKNOWN]
C -->|yes| D{protectionReasons present}
D -->|yes| K[PROTECTED preview]
D -->|no| E{lifecycleUnknown present}
E -->|yes| U
E -->|no| S[SAFE]
P --> Z[Exclude from deletion candidates]
K --> Z
U --> Z
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
codeant-ai
Bot
added
the
size:XXL
🏁 CodeAnt Quality Gate ResultsCommit: ✅ Overall Status: PASSEDQuality Gate Details
|
![codescene-access[bot]](https://avatars.githubusercontent.com/in/3755875?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
[check-pr-size] PR size is over the hard tier (normal profile): 18 files, 2530 meaningful lines, 8 commits — limit ≤20 files / ≤1200 lines / ≤10 commits. Consider splitting into smaller, independently reviewable PRs. |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
![codeant-ai[bot]](https://avatars.githubusercontent.com/in/646884?s=60&v=4){kind=small}
CodeAnt PR Risk: Medium Risk
Assessed commit: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0133481e44
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Classification: a preview without a usable commit SHA is UNKNOWN because it cannot be compared with main or PR heads; a preview without a creation time is ranked out of the newest-three selection instead of corrupting the sort; a deployment rolling to Production is Production-class; previews named in a reviewed hold file (.github/vercel-retention-holds.json) are kept as review or retest evidence. Authority: the gate is the push-triggered run on main for the exact SHA, so a manual or tag run cannot stand in for it, and a run list as long as the request limit is treated as truncated. Provider pages without pagination metadata are malformed instead of final. Pass: both modes print one manifest line per preview before the summary. The post-proof re-asserts the anchored gates and that Production is still the same READY deployment. A per-machine lock stops a second concurrent reconcile before it reads anything and is always released. The entry file loads only built-ins and the lock before the source check, so no runner module that reads or deletes is loaded from an untrusted source; the doc states that this guards against an accidental checkout, not a hostile one. The fixture is table-driven to keep each function small.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
- Configuration used: Repository: qnbs/WorldScript-Studio/.coderabbit.yaml
- Review profile: CHILL
- Plan: Essentials
- Run ID:
56fe5617-a1d2-42b8-bf65-05cf8e6b455a
📒 Files selected for processing (17)
.github/vercel-retention-holds.jsonCHANGELOG.mdREADME.mddocs/VERCEL-RETENTION-RUNNER.mdpackage.jsonscripts/vercel-retention/apply.mjsscripts/vercel-retention/authority.mjsscripts/vercel-retention/classify.mjsscripts/vercel-retention/cli.mjsscripts/vercel-retention/holds.mjsscripts/vercel-retention/lock.mjsscripts/vercel-retention/provider.mjsscripts/vercel-retention/scope.mjsscripts/vercel-retention/util.mjstests/unit/tooling/support/vercelRetentionFixture.mjstests/unit/tooling/vercelRetention.test.mjstests/unit/tooling/vercelRetention.test.ts
Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 2 reviews per hour.
|
@CodeAnt-AI review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f2770d8e93
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
CodeAnt Nitpicks2 code suggestions1.
|
Validate every open-PR, alias and deployment record before classification, protect branches of open fork PRs and failed previews of living branches, treat custom deployment targets as unknown, read page-boundary ties, take the lock atomically, print the manifest before the first write, and make the post-proof re-check the anchor, kept branch aliases and deferred IDs.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
@CodeAnt-AI review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4820471a31
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…o the retention post-proof Validate page cursors against their own page, treat a malformed PR identifier as unknown, settle an alias removal that failed after it landed by re-reading, and compare the alias bindings and settled states of deferred previews and the set of Production-class deployments in the post-proof.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
@CodeAnt-AI review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a70a625c5a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…identity and the final PR snapshot Check the Production deployment's own identity and the origin remote, treat a non-string branch and an alias host the removal would refuse as unknown, keep a displaced lock when its put-back fails, and make the post-proof require the exact planned Production-class set and re-classify deleted records against the final PR and branch snapshot.
|
@CodeAnt-AI review |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
- Configuration used: Repository: qnbs/WorldScript-Studio/.coderabbit.yaml
- Review profile: CHILL
- Plan: Essentials
- Run ID:
11d5cff8-364e-47c1-8711-d664b8860585
📒 Files selected for processing (11)
docs/VERCEL-RETENTION-RUNNER.mdscripts/vercel-retention/apply.mjsscripts/vercel-retention/authority.mjsscripts/vercel-retention/classify.mjsscripts/vercel-retention/cli.mjsscripts/vercel-retention/lock.mjsscripts/vercel-retention/provider.mjstests/unit/tooling/support/vercelRetentionFixture.mjstests/unit/tooling/vercelRetention.test.mjstests/unit/tooling/vercelRetention.test.tstests/unit/tooling/vercelRetentionSafety.test.mjs
Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 1 review per hour.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e290d57e5c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…predicate Rank open-PR retention slots from recognized plain previews only, accept a creation time only when it is a plausible instant, treat any environment id or unrecognized promotion substate as unknown, and apply the authorizing Production predicate again in the post-proof.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
@CodeAnt-AI review |
There was a problem hiding this comment.
Gates Passed
See analysis details in CodeScene
Quality Gate Profile: The Bare Minimum
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.
# Conflicts: # CHANGELOG.md # README.md
1 participant
User description
Summary
Adds a reviewed, versioned runner for the post-merge Vercel Preview retention pass, replacing the session-local scratchpad script. Part of #627.
pnpm run vercel:retention:dry-runis read-only.pnpm run vercel:retention:reconcileruns the full pass.Why
The first ordinary pass under the maintainer's retention decision needed a manual one-off command because the only executable was a temporary file under
/tmp. A durable runner removes that dependency. A shell allow rule is not a safety boundary, so the safety lives in the runner.What a pass does
CI / CDandCodeQL SASTrun of the SHA is a completed success, Production isREADYon it, the three canonical aliases are bound to it, canonical HTTP is 200, and the project identity matches fixed constants.SAFE,PROTECTEDorUNKNOWNin a pure function. Production-class deployments (production target, promoted, rollback candidate) never enter the classification.reconcileonly: deletes at most 25 per pass. Before each ID it re-readsmain, the gates and the whole provider state and re-classifies; a changed verdict only defers that ID. A branch alias of a gone branch goes first, after it is shown to bind to that deployment. Writes are never retried; a failed or unconfirmed write stops the pass.reconcilerefuses to run unlessscripts/vercel-retention/and the two package scripts equal the reviewedorigin/mainand are clean, so an edited or unmerged checkout cannot delete anything.Tests
172 cases against a simulated provider: authority (main moving at three points, push-only run gates and truncated run lists, five Production faults), pagination and read failure (including missing pagination metadata), project identity, Production-class protection, open-PR retention for 1/2/3/5 previews, unknown-metadata variants (commit SHA, creation time, state, identity), the explicit hold file, ROLLING and promoted Production-class protection, the per-preview manifest, reclassification before deletion, the post-proof Production re-check, the per-machine lock, refused/ambiguous/unconfirmed deletions, alias failure, batch limit, idempotence, redaction, source trust, argument-free entry points and ID/host validation. A live dry-run against the real project reports the same inventory as the hand-verified pass (42 Production, 2 rollback candidates, 0 unknown, 0 writes).
Not changed
No permission rule is installed and none is requested here; the exact command is documented as the one narrow rule the maintainer may choose to set. No change to the retention policy text (the Active-mode supersession is #1031), to tag, publish or Gate 7 authority, or to GitHub Pages record handling. Orphan aliases are reported by the provider but not removed.
Summary by Sourcery
Introduce a reviewed, versioned Vercel Preview retention runner that inventories and conservatively reconciles obsolete deployments with strict safety gates and post-run proof.
New Features:
Enhancements:
Build:
Documentation:
Tests:
Chores:
Summary by cubic
Replaces the temporary scratchpad script behind the Vercel Preview post-merge retention pass with a versioned runner under
scripts/vercel-retention/(issue #627).pnpm run vercel:retention:dry-runis read-only;pnpm run vercel:retention:reconcileruns the full pass, and both refuse any argument so an allow rule cannot be widened by appending text.Safeguards
READY, canonical aliases bound, canonical HTTP 200) and the fixed project and origin identity, reads every page of deployments and aliases, validates every record before classification, and classifies each preview asSAFE,PROTECTEDorUNKNOWNin a pure function. A manual or tag run never stands in, a run list at the request limit counts as truncated, a page without pagination metadata as malformed, and each page cursor is checked against its own page.PROTECTEDcovers canonical or live-branch aliases, an open PR's newest three plain previews (ranked only from recognized plain previews with plausible creation times),main/open heads, and previews named in.github/vercel-retention-holds.json; missing, contradictory or foreign metadata (commit SHA, creation time, state, rollback flag, identity, PR identifier, environment id, promotion substate) makes a previewUNKNOWN.reconciledeletes at most 25 previews per pass, re-readsmain, the gates and the whole provider state before each ID, removes the branch alias of a gone branch first (settling a removal that landed despite a reported failure), never retries a write, and ends with a post-proof that re-asserts the gates, Production, the full Production predicate, kept branch aliases and deferred IDs together with their alias bindings, settled states and the Production-class set; one pass runs per machine.reconcileruns only when the runner, hold file and package scripts equal the reviewedorigin/main; the entry file loads only built-ins and the lock before that check.Covered by the node:test suites against a simulated provider, plus a live dry-run matching the hand-verified inventory.
Written for commit b0eeeca. Summary will update on new commits.
Summary by CodeRabbit
CodeAnt-AI Description
Add a reviewed Vercel Preview cleanup runner
What Changed
Impact
✅ Preview deletions reviewed before cleanup✅ Production and active pull request previews retained✅ Uncertain cleanup stops before further deletions💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.