Skip to content

perf: faster hooks, diff, Codex and recovery fixes (follow-up to #12) - #13

Merged
maxgfr merged 9 commits into
mainfrom
issue-12-perf-followup
Oct 2, 2026
Merged

maxgfr merged 9 commits into
mainfrom
issue-12-perf-followup

Conversation

@maxgfr

@maxgfr maxgfr commented Oct 2, 2026 •

Copy link
Copy Markdown
Owner

Follow-up to #12. I audited every finding against 0.6.0 and found a few gaps. Profiling also found two costs paid on every call. The branch then went through a two-axis review (standards, spec), and its findings are fixed here.

Performance

Each hook is a new process, so the fixed cost of every call matters most.

  • Foundation no longer loads at startup. Through dispatch2, ctrlc linked Foundation and the Objective-C runtime. Loading them cost about 0.6 ms on every invocation, even for hooks that compress nothing. SIGINT, SIGTERM and SIGHUP now set the cancellation flag through a direct sigaction with SA_RESTART, replacing whatever disposition the caller left, as ctrlc's set_handler did. Non-Unix builds keep ctrlc.
  • Cheaper regex compilation. The compact-v3 patterns used Unicode \s, \d and case folding, which their ASCII vocabulary doesn't need. Compiling them took about 1.1 ms; it now takes about 0.5 ms. Java frames keep Unicode \w, and a test pins that.

Measured on macOS arm64:

0.6.0 this branch
hook, small output 2.7 ms 2.1 ms
hook, cargo test output 4.6 ms 3.4 ms
bench/performance.py --stress −3 % to −19 % median, −4 MB RSS on every case

The compact-v1/v2 output is byte-identical to 0.6.0, artifact IDs included. Both reviews checked this on 200 fixture/profile/version combinations. The performance bench also checks byte identity between its arms. Exit codes are unchanged.

Fixes

  • Diffs (v3). With Git's default three lines of context, a hunk header sits four lines above its first change. That is one line past the glue reach, so the header ranked last and was dropped. A hunk header now glues to the first change below it and ranks with it.
  • git log -p (v3). Commit messages folded by template, so subjects that differ only by a number collapsed into one similar= line. That contradicts design.md. They now fold only with identical text.
  • Codex coverage. git diff HEAD~1 and rg "\d+" src stayed native.
    • A ~ inside a word is literal, and so is a backslash before an ordinary character inside double quotes. Both now parse.
    • A tilde that would expand and \" or \\ are still rejected.
    • I checked the parsed words against sh, bash and zsh with default options. zsh's EXTENDED_GLOB would treat a mid-word ~ as a glob, the same caveat the subset already accepts for #.
  • Codex coverage, more runners. npx -y/--yes <known tool> and yarn <known tool> (yarn jest, yarn tsc) are now recognized. Unknown tools, npx -p and yarn subcommands such as add stay native.
  • Store growth. Paging or searching a blob whose view is all of it (typically one long line) stored that original again in a schema-1 artifact for every distinct recovery: 12 MB for two recoveries of a 6 MB JSON line. Such a result now names the blob (schema 2), but only when the blob is actually stored. The review found that a file read locally after a repository scan names a blob that was never kept, and a test covers that case.

Intended v3 output changes

Only the diff ranking changes. The other fixtures and stress inputs are identical in v3.

  • git log -p, git profile.
    • 14 distinct commit subjects are shown instead of 2.
    • Changes, hunks and file headers are the same.
    • 35 index metadata lines no longer fit, so display_complete becomes false (omitted_units 0 → 35).
  • git log -p, default 4 KB budget.
    • Distinct subjects (tier 2) now come before changes repeated across commits (tier 1), as design.md specifies.
    • 14 subjects instead of 2; 15 changes instead of 20; 10 hunks instead of 13.
  • Unified diff, default profile.
    • Selection order shifts slightly: a +++ line that repeats the file named by diff --git and four index lines are no longer shown (omitted_units 8 → 13).
    • Changes and hunks are the same.
  • Recovery artifact IDs. For a recovery whose only record is a whole stored blob, the artifact ID changes (schema 2). This is outside the compact-v1/v2 invariant.

Tested and left as is

  • A code line such as src/lib.rs: pub enum Error { still ranks as a diagnostic in search output. Turning ranking off for the search profile fixes that line, but an rg over logs then shows 1 of 3 errors instead of 3.

Known limits, not addressed

  • Large outputs. Over 250k lines (V3_MAX_LINES), output passes through unchanged. Raising that needs less memory per line first, starting with the full input copy in automatic_with.
  • Recovery path. The view header still says scopelet expand. The absolute path is in the session guidance, not in every header.
  • Native on purpose. FOO=1 cmd with a variable name off the allow-list stays native, and so does a pipeline that already ends in tail.

Verification

  • cargo fmt --check and cargo clippy --all-targets --locked -- -D warnings.
  • cargo test --locked: 178 tests.
  • python3 scripts/check_skill.py.
  • python3 -m unittest discover -s bench -p 'test_*.py'.
  • bench/content.py: every fact is visible or recoverable.

Closes #12

maxgfr added 9 commits October 2, 2026 19:05
Every hook is a new process, so the compact-v3 patterns are compiled on
every compressed call. Unicode whitespace, digits and case folding made
them several times larger to compile than their ASCII vocabulary needs.
Indentation, digits and case folding are now ASCII; a Java frame keeps
Unicode \w, since Java identifiers may be Unicode, and a test pins it.

Outputs are byte-identical on the fixture corpus and stress inputs in
v1, v2 and v3.

Refs #12
On Apple platforms ctrlc links Foundation and the Objective-C runtime
through dispatch2. Loading them cost about 0.6 ms on every invocation,
hooks included, a quarter of a hook that compresses nothing.

SIGINT, SIGTERM and SIGHUP now set the cancellation flag through a
direct sigaction with SA_RESTART, replacing any disposition the caller
left, as ctrlc's set_handler did. ctrlc remains for non-Unix builds. A
test signals the binary, with and without ignored signals, and expects
run --auto to stop its command and exit 130.

Refs #12
Git's default three lines of context put a hunk header four lines above
its first change, one past the glue reach: the header ranked last and a
diff showed changes without the line numbers locating them. A hunk
header now glues to the first change below it at any distance and ranks
with it.

Commit messages of git log -p folded by template, so messages differing
by a number collapsed into one similar= line, contrary to design.md:
they now fold only with identical text, like changes. On the git log -p
fixture each commit's subject replaces index metadata lines.

Refs #12
The shell subset rejected any ~ and any backslash inside double quotes,
so git diff HEAD~1 and rg "\d+" src stayed native. sh, bash and zsh
only expand a tilde at the start of a word or after an assignment's =
or :, and inside double quotes keep a backslash literal before an
ordinary character. Those forms are now parsed to the words every shell
gives them; a tilde that would expand and a backslash before " or \
are still rejected.

Refs #12
Paging or searching a single-line blob shows its only line as one record,
and the view's schema-1 artifact stored that line again: recovering a
6 MB JSON line twice stored 12 MB. A recovery whose only record is a
whole text original now names the blob (schema 2), as compact-v3
artifacts do; a hash check settles that the text is the whole blob.
Two recoveries of a 2 MB line now store kilobytes and their artifacts
still expand to the exact line.

Refs #12
A repository scan keeps only the originals of files that matched, and a
later search reads an unmatched file from its unchanged local copy. That
record still names the file's blob, so the schema-2 shortcut wrote an
artifact referring to a blob the store never had, which then failed to
expand. The shortcut now requires the blob to be stored, and compares
the record with what the text parser reads back from it instead of
repeating that parser's fields.

Refs #12
A fixed 500 ms wait could signal the binary before its handler was
installed on a loaded machine. The test now waits until the wrapped
sleep is a child of the binary, which happens after the handler is set.

Refs #12
design.md still described hunk headers as glued within three lines and
limited schema-2 recoveries to single-line blobs; the shell subset
rejects a tilde after any = or :, not only an assignment's.

Refs #12
npx -y only skips the install prompt, and yarn <bin> runs a package
binary as npx does, so npx -y tsc and yarn jest finish and report an
outcome like their recognized forms. Unknown tools, npx -p and yarn
subcommands such as add stay native.

Refs #12
@maxgfr maxgfr changed the title perf: faster hooks, diff and recovery fixes (follow-up to #12) perf: faster hooks, diff, Codex and recovery fixes (follow-up to #12) Oct 2, 2026
@maxgfr
maxgfr merged commit bbe5b2a into main Oct 2, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimisation globale : couverture, compression, récupération, store, perf

1 participant