Skip to content

feat: add code-review-calibration skill (non-default) - #512

Draft
rbren wants to merge 2 commits into
mainfrom
add-code-review-calibration-skill
Draft

rbren wants to merge 2 commits into
mainfrom
add-code-review-calibration-skill

Conversation

@rbren

@rbren rbren commented Aug 25, 2026

Copy link
Copy Markdown
Member
  • A human has tested these changes.

Why

The existing code-review skill reviews from general principles, so its feedback is much the same in every repository. What a reviewer actually needs is knowledge of how this codebase breaks and what "correct" looks like here - and that knowledge is recoverable from the repo's own commit log, bug fixes, reverts, and past PR reviews.

This skill generates that calibrated reviewer instead of hand-writing it.

Summary

  • Adds skills/code-review-calibration/SKILL.md, a skill that investigates a repository in two halves - how it breaks (hotspots, reverts, SZZ fix-inducing commits, temporal coupling) and how it works (conventions, testing strategy, CI boundary, ownership and blast radius) - then writes or updates .agents/skills/code-review.md with failure patterns that each cite two or more SHAs.
  • Registers it in marketplaces/openhands-extensions.json under code-quality. defaultEnabled is deliberately omitted, so the skill is opt-in from the catalog UI rather than seeded into new workspaces.
  • Regenerates skills/index.js and the README catalog table.

Issue Number

How to Test

npm run build:skills                      # regenerates cleanly, no diff
python scripts/sync_extensions.py --check # only the pre-existing issue-duplicate-checker warning
uv sync --group test && uv run pytest tests/test_skills_catalog.py tests/test_catalogs.py -q

Confirm the skill is registered but not default-enabled:

node -e "import('./skills/index.js').then(m=>{
  const s=m.SKILLS_CATALOG.find(x=>x.name==='code-review-calibration');
  console.log('found:', !!s, 'defaultEnabled:', s.defaultEnabled);
  console.log('in defaults:', m.DEFAULT_ENABLED_SKILL_NAMES.includes('code-review-calibration'));
})"

Expected: found: true defaultEnabled: undefined and in defaults: false, with DEFAULT_ENABLED_SKILL_NAMES unchanged at 11 entries.

To exercise the skill itself, enable it in a workspace with a repo that has real history and ask for a calibrated code review skill; it should produce .agents/skills/code-review.md where every failure pattern cites SHAs from that repo.

Video/Screenshots

n/a - skill content and catalog metadata only.

Notes

  • Em dashes in the source were converted to plain hyphens per the punctuation convention in AGENTS.md.
  • Per AGENTS.md, "defaultEnabled": false is never written - absence is how a skill stays off - so the entry simply omits the key.
  • The not in any marketplace: ./plugins/issue-duplicate-checker warning from the sync script predates this branch and is unrelated.

This PR was created by an AI agent (OpenHands) on behalf of @rbren.

Adds skills/code-review-calibration, a skill that mines a repository's
own commit history, bug fixes, reverts, conventions, testing strategy,
CI coverage, and past PR reviews to generate a calibrated
.agents/skills/code-review.md with cited failure patterns.

Registered in marketplaces/openhands-extensions.json without
defaultEnabled, so it stays opt-in from the catalog UI.

Co-authored-by: openhands <openhands@all-hands.dev>
@rbren rbren changed the title Add code-review-calibration skill (non-default) feat: add code-review-calibration skill (non-default) Aug 25, 2026
CI requires every skill directory to ship a README.md and a
.plugin/plugin.json with vendor symlinks so Codex and Claude Code can
load it.

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions github-actions Bot added the type: feat A new feature label Aug 25, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: feat A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant