Skip to content

Add a fast model for commit messages and split plans - #42

Closed
rafaeelricco wants to merge 3 commits into
mainfrom
rafaeelricco/fast-commit-model
Closed

rafaeelricco wants to merge 3 commits into
mainfrom
rafaeelricco/fast-commit-model

Conversation

@rafaeelricco

Copy link
Copy Markdown
Owner

Motivation

Make commit generation faster. Every message and split plan waits on the main model at its configured effort, even though writing a commit message is simple summarization that a smaller model handles well.

This branch sits on top of #37, so the PR also shows #37's two commits until #37 merges. Only the last commit, "Add a fast model for commit messages and split plans", belongs to this PR.

What's New

Fast Model Config

  • A new optional fast_model config field (a model id from the same provider and sign-in as ai). Older config files without it load as Nothing and behave exactly as before; commit model keeps it and commit setup resets it.
  • When it is set, Commit.create swaps it in with withModel + withMinEffort, so the plain message, Regenerate, Adjust and split plans (including split-plan Regenerate in Split) all run on it at the provider's lowest effort. commit branch keeps the main model.

commit fast-model Command

  • Picks a model from the provider's list, or "Use the main model" to turn it off; cancelling saves nothing. It saves the sign-in that resolveProvider just refreshed, so running it never writes stale OAuth tokens back.
  • Listed in commit --help, the README and the dispatcher in index.ts; commit doctor shows it after the main model (claude-sonnet-5 (medium effort), fast: claude-haiku-4-5).

Anthropic Reasoning Retry

  • Claude Haiku 4.5 rejects adaptive thinking and effort, which the Anthropic path always sent. A 400 whose message mentions either now gets one retry without both, on the API-key and setup-token paths alike, and the commit note reports "provider default" effort. Other 400s are not retried. This mirrors the retry-without-effort that the OpenAI and xAI paths already have.

Model Selection Flow

flowchart TB
    A["Commit.create()"] --> B["resolveProvider(config)"]
    B --> C{"fast_model set?"}
    C -->|no| D["main ai, configured effort"]
    C -->|yes| E["withModel + withMinEffort"]
    D --> F["messages, Adjust, split plans"]
    E --> F
    F --> G{"Anthropic rejects thinking/effort?"}
    G -->|no| H["reply"]
    G -->|yes| I["retry once without them"]
    I --> H
Loading

Changed Files

File Change Type Summary
src/domain/config/config.ts Modified Optional fast_model field.
src/cli/commit.ts Modified Commit.create swaps in the fast model at lowest effort.
src/cli/fast-model.ts Added commit fast-model pick / clear command.
src/cli/parser.ts, index.ts Modified fast-model command, help text, dispatch, header notes.
src/cli/doctor.ts Modified Shows the fast model next to the main one.
src/cli/setup.ts Modified New configs start with no fast model.
src/infra/llm/anthropic.ts Modified One retry without thinking/effort when the model rejects them.
src/domain/llm/auth-resolver.ts Modified Header lists FastModel among the refreshing commands.
README.md Modified commit fast-model usage and table row.
test/** Modified/Added Config default, fast model reaching commit and split calls, command cases, Anthropic retry; fixtures gain fast_model: Nothing().

Testing & Feedback

Check Commit.create in src/cli/commit.ts: every router call in commit and split should see the fast config, and commit branch should not. In src/infra/llm/anthropic.ts, check that the retry only fires on 400s that mention effort or thinking. The new commit, split and Anthropic-retry tests fail on the pre-change code.

Not verified: the exact wording Anthropic returns when Haiku 4.5 gets effort or adaptive thinking. To try it for real:

  • Run commit fast-model and pick claude-haiku-4-5, then run commit on a staged change. The commit note should show claude-haiku-4-5 with provider default effort.
  • Stage two or more files with split_commits on and confirm the split plan also comes from the fast model.
  • Run commit fast-model again, choose "Use the main model", and confirm commit doctor no longer shows a fast model.

If you find any bugs or have recommendations for improvements, please open an issue and assign it to me.

- Call `generateContent` with `withDefaultMinEffort(config)` in `generateSplitPlan`, so an explicit effort reaches the provider and an unset one still falls back to the lowest level.
- Drop the unused `withMinEffort` import from `src/domain/llm/router.ts` and update the module header to list split plan under `withDefaultMinEffort`.
- Assert a configured `high` effort reaches the provider and an unset effort defaults to `low` in the split-plan router tests.
- Replace the "internally decide" and stacked "ONLY / Do NOT" output lines in the conventional, imperative, and custom prompts with one line explaining that the reply becomes the commit message verbatim.
- Capitalize the imperative prompt's style examples and negative example so they match its capital-letter rule, and make the example body bullets complete imperative sentences.
- Label the split-plan examples as illustrative so their imperative messages do not override the active convention.
- Limit the branch-name "forbidden anywhere" list to words from the instructions, so area prefixes like `cli` stay usable, and drop the `<synthesis_protocol>` steps while keeping the no-subset rule.
- Pass `commit_convention` and `custom_template` through `refineCommitMessage` into `getRefinePrompt`, which now states the configured convention and the size-based format instead of inferring the convention from the current message.
- Cover the refine prompt's use of a custom template with a unit test and update the refine call sites in the router tests.
- Add an optional `fast_model` to the config; files without it load as `Nothing` and keep using the main model, and `commit setup` resets it.
- Swap `fast_model` into `Commit.create` at the provider's lowest effort, so messages, Regenerate, Adjust and split plans (including split Regenerate) use it while `commit branch` keeps the main model.
- Add `commit fast-model` to pick a model from the provider's list or go back to the main model, saving the sign-in refreshed by `resolveProvider`.
- List the command in `commit --help`, the README and the dispatcher, and show the fast model in `commit doctor`.
- Retry an Anthropic request once without adaptive thinking and `effort` when the model rejects them, as Claude Haiku 4.5 does, and report its effort as "provider default".
- Cover the config default, the fast model reaching the commit and split-plan calls, the command's pick, clear, cancel and refreshed-sign-in cases, and the Anthropic retry with unit tests.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-27T16:23:57.662373Z 1739ec9 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@rafaeelricco

Copy link
Copy Markdown
Owner Author

Closing: keeping only the prompt updates, which are in #37. The fast-model config isn't wanted.

@rafaeelricco
rafaeelricco deleted the rafaeelricco/fast-commit-model branch September 27, 2026 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant