Repository navigation
context_pack: --eval hit rate against merged PRs (Part of #5521) - #6354
Conversation
…s before= cutoff (Part of #5521) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
No description provided. |
|
No description provided. |
|
PR #6354 @ 30d93d8: 2 failed check(s) == full-suite (failure) -- main == full-suite part 3/3 (failure) -- real |
…t of #5521) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
No description provided. |
|
PR #6354 @ a020053: 1 failed check(s) == full-suite part 1/3 (failure) -- real |
…doc); regenerate the list (Part of #5521) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
CI shepherd: the head of this PR (8100510) is not the one a maintainer labelled |
|
No description provided. |
|
No description provided. |
|
No description provided. |
|
No description provided. |
Part of #5521.
What
context_pack.py --eval Nis the committed, reproducible hit-rate measurement the acceptance item asked for (#6145's was a scratch harness). For the newest N issue-closing squash commits onorigin/main("Part of/Closes #I" and "(#PR)" in the subject) it rebuilds the pack as of just before the PR (prior_files(..., before=sha)excludes the PR itself and later commits), then reports pack tokens and how many of the PR's changed files the pack names, split into code files (tests included; docs, data, workflows, agent notes excluded) and any file. Doc paragraph added todocs/agents/self-scan-orientation.md. Plan said to filter to burned estates; dropped: this is infra, and the sample is whatever churn closed.Hit rate (24 newest issue-closing PRs, DB at main 8437f15,
--eval 24, ~12 s)Largest pack 829 tokens (cap 3000).
Misses are mostly epics with no earlier PR yet (#5051, #6072, #5116 style) or PRs whose files the issue text never names. The DB is at HEAD, so a file a PR deleted or renamed cannot be named. Full per-issue table:
Issues: 24. Pack names >=1 changed file: 13/24; >=1 changed source file: 12/24; every changed source file: 8/24.
Checklist
Checklist: done [c:530ba3ea]
Class / Sibling gaps
Class: "does the pack point a worker at the files its PR changed" (path, symbol, class, language stem and earlier-PR resolution) plus its measurement; this PR adds the measurement and the
before=cutoff, no ranking change. Sibling gaps (none needs a burned-estate program): promoter-botchurn workerwiring ([c:8c9805b3], other repo); orchestrator per-commit DB refresh before launch; "(Delta Mode)" label on full scans atgalaxyscope.py:2024(engine edit, golden check); recall for epics with no earlier PR (a ranking limit, no mechanism yet).Gates
pr_gates --fast7/7;test_context_pack.py16 pass (also plainpytestfrom/tmp);test_source_reads.py+test_generated_branch.pypass. No parsing change, so no golden bless.🤖 Generated with Claude Code