Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion automations/bundle-index.js

Large diffs are not rendered by default.

12 changes: 6 additions & 6 deletions automations/catalog/github-pr-reviewer/manifest.json
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
{
"id": "github-pr-reviewer",
"version": "1.7.0",
"version": "1.8.0",
"name": "GitHub code review",
"category": "Code review",
"description": "Review requested pull-request heads, resuming an outstanding reviewer request on a schedule once its checks are green, draining the backlog oldest-request first under a small per-scan new-conversation bound, record exact-head decisions, stop early on out-of-scope changes, and optionally request a code-aware human maintainer after approval or a maintainer-decision scope stop.",
"description": "Review requested pull-request heads, resume an outstanding reviewer request on a schedule once its checks are green, review a bounded rotating window of open non-draft pull requests that nobody requested once their current head is green and unreviewed, drain the backlog oldest-first under a small per-scan new-conversation bound, record exact-head decisions, stop early on out-of-scope changes, and optionally request a code-aware human maintainer after approval or a maintainer-decision scope stop.",
"requires": {
"integrations": {},
"features": [
Expand All @@ -13,7 +13,7 @@
},
"popularityRank": 100,
"estimatedSetupMinutes": 4,
"exampleImplementation": "A scheduled scan of the trigger label and of open non-draft PRs holding an outstanding reviewer request, or a GitHub reviewer-request event, selects a PR head and submits an idempotent subject turn. The scheduled scan orders eligible heads by the oldest outstanding request and starts at most a small configurable number of new conversations per scan across every repository. The profile-backed agent clones that exact head in its runtime, applies an early repository-ownership and product/architecture scope gate, then reuses the established review workflow, runs appropriate tests, and publishes a readable native review. A native approval, or a maintainer-decision scope stop, can then be handed to a code-aware human maintainer.",
"exampleImplementation": "A scheduled scan of the trigger label, of open non-draft PRs holding an outstanding reviewer request, and of a bounded rotating window of open non-draft PRs nobody requested whose current head is green and unreviewed, or a GitHub reviewer-request event, selects a PR head and submits an idempotent subject turn. The unrequested window resumes from a per-repository position kept in the Automation KV store, so each scan reads a bounded number of pull requests while still covering the backlog over time, and explicit requests are always examined. The scheduled scan orders eligible heads with explicit requests first and then oldest-first, and starts at most a small configurable number of new conversations per scan across every repository. The profile-backed agent clones that exact head in its runtime, applies an early repository-ownership and product/architecture scope gate, then reuses the established review workflow, runs appropriate tests, and publishes a readable native review. A native approval, or a maintainer-decision scope stop, can then be handed to a code-aware human maintainer.",
"impact": {
"basis": "completed-runs",
"one": "1 PR review sweep completed",
Expand All @@ -28,7 +28,7 @@
"schedule": {
"type": "cron",
"label": "Check frequency",
"help": "How often to look for newly labelled pull requests and for outstanding reviewer requests whose checks have not yet finished.",
"help": "How often to look for newly labelled pull requests, outstanding reviewer requests whose checks have not yet finished, and open non-draft pull requests nobody requested that still need a review.",
"default": "*/5 * * * *",
"required": true
},
Expand Down Expand Up @@ -68,7 +68,7 @@
"triggerLabel": {
"type": "text",
"label": "Trigger label",
"help": "Only pull requests carrying this label are reviewed.",
"help": "Pull requests carrying this label are reviewed. An open, non-draft pull request that nobody requested is also reviewed once its current head is green and carries no review yet, examined a bounded rotating window at a time so one scan reads only a slice of the backlog.",
"default": "openhands-review",
"required": true,
"constraints": {
Expand Down Expand Up @@ -135,7 +135,7 @@
}
},
"bundle": {
"version": "1.7.0",
"version": "1.8.0",
"entrypoint": "python3 worker.py",
"timeout": 300,
"files": {
Expand Down
11 changes: 11 additions & 0 deletions skills/github-pr-reviewer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,17 @@ This skill is activated by:
- Resumes an outstanding reviewer request on the next scheduled scan once the
requested head's checks are green, so a request that arrives during CI is not
lost; the explicit-request event path is kept for event-only deployments
- Reviews open, non-draft PRs that nobody requested on a scheduled scan once
their current head is green and carries no review yet, keyed by
repository/PR/head so a repeat scan never duplicates a conversation or review
and a changed head becomes eligible again
- Examines a bounded, rotating window of that unrequested backlog per scan
(10 PRs per repository), remembering its position in the Automation KV store so
the next scan resumes past it; explicit requests and trigger labels are always
examined, never skipped behind the window
- Posts no managed gate comment for an unrequested PR that is merely red or
pending - the comment answers an explicit request, so a scan over a large
backlog cannot storm the PRs with comments
- Names the retry the deployment actually has in the waiting comment: a
scheduled scan where a cron trigger exists, and removing and re-requesting the
bot where only the event trigger does
Expand Down
56 changes: 48 additions & 8 deletions skills/github-pr-reviewer/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,41 @@ symbolic base branch carries no branch rules of its own:
`review_requested` event, so repeated scans reuse one conversation and one
review instead of creating duplicates, and a PR whose request was answered or
withdrawn simply drops out of `requested_reviewers`.
- The scheduled scan also reviews open, non-draft PRs that **nobody requested**
and that carry no trigger label, once their current head has no completed
review by this account. That is what keeps reviewing PRs after the outstanding
requests are exhausted. The delivery key is the stable
`scan:{repository}:{number}:{head}`, so repeated scans reuse one conversation
and one native review, and a changed head becomes eligible again under its new
SHA.
- The unrequested part of a scan is a **bounded, rotating window**, not the whole
backlog. Classifying one unrequested head costs a review read plus the
exact-head check and workflow reads, so examining every open PR in one scan
spent the run's API budget in the largest repository and announced every red or
pending head. A scan examines at most `SCAN_WINDOW` (10) unrequested PRs per
repository and stores where the window ended in the automation service's own
KV store, under a per-repository `review-scan:{owner}__{repo}` key, so the next
scan resumes past that point and the backlog is covered fairly over several
scans. The window wraps at the end of the backlog. When the KV store is not
available (a local run) the position is kept in memory, so the scan still
rotates within the run.
- An explicit `all-hands-bot` review request or a trigger label is **never**
subject to the window: every explicit candidate is examined on every scan,
whatever the stored position is, so a request is not delayed behind the
rotation.
- A gate stop on an **unrequested** head posts no managed comment. A PR nobody
asked about that is merely red or pending is simply skipped without starting an
agent, which is what keeps a scan over a large backlog from posting a gate
comment on every PR. The managed comment is the answer to an explicit request,
so a requested or labeled head that is red or pending still gets its
explanation.
- Workflow runs are read as well as check runs, because a workflow can fail
before creating any check run - a workflow-level error, or a `pull_request`
run whose jobs never start. Such a run leaves a failed check suite with no
check runs under it, so the commit's check-run rollup and `gh pr checks` both
report success and only the workflow run reveals the red CI. A workflow run
whose check suite already reported check runs is left to those runs, so a
workflow is never counted twice.
- Runs attributed to any other (obsolete) head SHA are ignored, so a stale
failure cannot block the push that fixed it. This applies to workflow runs
too.
Expand Down Expand Up @@ -128,18 +163,23 @@ is `2`, and the rendered `config.json` overrides it through the
`max_new_per_run` key that `github-issue-to-pr` and `gitlab-issue-to-mr` already
use.

- Eligible candidates are ordered by the oldest outstanding `all-hands-bot`
`review_requested` event, then by repository, then by pull-request number, so
the oldest requests drain first and a later scan reaches the remainder.
- Eligible candidates are ordered deterministically: an explicit `all-hands-bot`
request (or a trigger label) is ordered before an unrequested PR, then by the
oldest `all-hands-bot` `review_requested` event for a requested PR or the PR's
own creation time (oldest first) for an unrequested one, then by repository,
then by pull-request number, so the oldest work drains first and a later scan
reaches the remainder.
- The bound counts the conversations a scan **starts**. A delivery that only
deduplicates or reports an already-running conversation reuses a runtime and
consumes no slot, so repeated scans make progress on the backlog instead of
re-spending the bound on work already in flight.
- Reaching the bound never cuts the scan short. The scan still evaluates the
exact-head checks of the remaining candidates, posts its waiting or blocked
gate comments, reconciles completed reviews, and runs the maintainer handoff.
A candidate whose checks are pending or failing starts no conversation and
consumes no slot, so it cannot block a later green candidate.
exact-head checks of the remaining candidates, reconciles completed reviews,
and runs the maintainer handoff. A candidate whose checks are pending or
failing starts no conversation and consumes no slot, so it cannot block a later
green candidate. Only an explicit request or a trigger label gets a waiting or
blocked gate comment; an unrequested head that is merely red or pending is
skipped silently.
- A dispatch that raises is reported and does not consume a slot or abort the
scan, so the candidates behind it are still considered.
- The explicit `review_requested` event path and the trigger-label scan are
Expand Down Expand Up @@ -459,7 +499,7 @@ The completion callback fires once for the whole run.
| Review paused with a failing-check comment | A current-head required check reported `failure`, `cancelled`, or `timed_out` | Fix the named checks and push; the review starts on the new head, or request `all-hands-bot` to review immediately |
| Review reported waiting on checks | A current-head required check is `queued` or `in_progress`, or has not reported yet | No action; a later scan or a new review request retries |
| Optional workflow failed but no review was paused | The failed workflow is not required, so the required-only scheduled gate ignored it | No action; only GitHub-required checks gate scheduled discovery |
| Only a few reviews start on a large backlog | The per-scan `max_new_per_run` bound (default 2) reached | No action; later scans drain the remaining oldest requests, or raise `max_new_per_run` if the deployment can hold more agents |
| Only a few reviews start on a large backlog | The per-scan `max_new_per_run` bound (default 2) reached | No action; later scans drain the remaining oldest requests and the bounded rotating window of unrequested PRs, or raise `max_new_per_run` if the deployment can hold more agents |
| Review result never posts | Conversation still running or stuck | Open the conversation link from the acknowledgement comment |
| Stale review suppressed | PR head SHA changed while the agent was reviewing | Re-apply the trigger label after the latest commit |
| Review arrives as a plain comment, not a review | Publishing failed, so the script posted the text as a fallback | Check that the token has Pull requests: Read and Write |
Expand Down
15 changes: 15 additions & 0 deletions skills/github-pr-reviewer/references/state-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,21 @@ store under the key `state:{owner}__{repo}` — for example
`AUTOMATION_KV_TOKEN` is injected into the run environment. Each automation has
its own isolated namespace.

The scheduled worker (`worker.py`) uses the same store for one more document, the
**unrequested-scan cursor**, under the key `review-scan:{owner}__{repo}`:

```json
{ "cursor": 20 }
```

`cursor` is the position in the repository's unrequested-PR backlog at which the
next scheduled scan's bounded window starts. Each scan examines at most
`SCAN_WINDOW` (10) unrequested PRs from that position, then writes the position
after the window, so successive scans rotate through the whole backlog instead of
reading one pull request per open PR. A cursor past the end wraps to the start.
This document is separate from `main.py`'s per-repository review state; a missing
or unreadable cursor simply starts the window at the beginning.

**Fallback (local/dev):** When the KV store is not available, the state is
written to a local JSON file at:

Expand Down
Loading
Loading