diff --git a/docs/MODEL-PRICING-AUDIT.md b/docs/MODEL-PRICING-AUDIT.md index b5c09f97..4ab96e8d 100644 --- a/docs/MODEL-PRICING-AUDIT.md +++ b/docs/MODEL-PRICING-AUDIT.md @@ -37,8 +37,9 @@ which the estimator does not model. GPT-6 models first and labels GPT-5.6 "Older"; OpenAI's Codex docs make Sol the default preset ([Codex models](https://learn.chatgpt.com/docs/models)). GPT-5.6 models remain listed for pins. -- Cross-host tiers pair by role and per-token price: GPT-6 Sol ↔ Sonnet 5 - ($2/$10), GPT-6 Astra ↔ Fable 5.1 ($10/$50), GPT-6 Luna ↔ Haiku 4.5. +- Cross-host tiers pair by role: GPT-6 Sol ↔ Sonnet 5 (balanced, $2/$10), + GPT-6 Astra ↔ Opus 5.5 (reasoning; Astra also pairs with Fable 5.1), GPT-6 + Luna ↔ Haiku 4.5 (fast). A Codex-driven seed runs reasoning work on Astra. - Routes seeded before this change are reported as diverged and keep their model until `ak x host refresh`. - Not added as retirements: OpenAI's Codex docs retire GPT-5.4 / 5.4 Mini diff --git a/docs/PROVIDERS.md b/docs/PROVIDERS.md index bf0adaf0..d9f2c19f 100644 --- a/docs/PROVIDERS.md +++ b/docs/PROVIDERS.md @@ -460,8 +460,19 @@ ak host pick --primary-host codex # make codex the lead; claude becomes th **Which host leads.** `--primary-host claude|codex` (default `claude`) chooses the primary. Codex-primary **mirrors** the default table below — codex takes the reasoning/review lead and claude becomes the alternate/escalation target — so the experience is ambidextrous -regardless of which CLI drives. `ak status` marks the primary and fails (not warns) if the -primary host is missing. +regardless of which CLI drives. Models pair by tier, so each host leads reasoning work on its +reasoning model and escalation steps up a tier either way: + +| Tier | Claude | Codex | +|---|---|---| +| reasoning | `claude-opus-5-5` | `gpt-6-astra` | +| balanced | `claude-sonnet-5` | `gpt-6-sol` | +| fast | `claude-haiku-4-5-20251001` | `gpt-6-luna` | + +With Codex leading, architecture, design, security-analysis and debugging run on `gpt-6-astra`, +and implementation and testing run on `claude-sonnet-5`, escalating to `gpt-6-astra`. Astra costs +more per token than Opus 5.5 ($10/$50 vs $4/$20), so Codex-led reasoning uses more Codex allowance. +`ak status` marks the primary and fails (not warns) if the primary host is missing. **OpenCode is explicit, not seeded or AQE-projected.** Enable it, then use `ak run` with either a persisted route or a run-local override: diff --git a/docs/adr/0002-activity-vocabulary-defaults-from-ruv-templates.md b/docs/adr/0002-activity-vocabulary-defaults-from-ruv-templates.md index 82c149d3..4fec7745 100644 --- a/docs/adr/0002-activity-vocabulary-defaults-from-ruv-templates.md +++ b/docs/adr/0002-activity-vocabulary-defaults-from-ruv-templates.md @@ -48,13 +48,14 @@ as such** wherever surfaced (an `ak` tag in the UI, a comment in the defaults ta Default *models* map to the host's appropriate tier (Opus for deep reasoning, Sonnet for review, a Codex model for execution) and are treated as **soft defaults** — see the "open question" on pinning live model IDs. -Tier is the pairing key, not the model id: `MODEL_CATALOG` spells `flagship`/`balanced`/`fast` +Tier is the pairing key, not the model id: `MODEL_CATALOG` spells `reasoning`/`balanced`/`fast` identically on both hosts so primary-host mirroring can map a route to its counterpart's equivalent. As of 2026-09-23 execution routes to `gpt-6-sol` (balanced) and mechanical work to `gpt-6-luna` (fast), and deep reasoning routes to `claude-opus-5-5`. These are agentic-kit default preferences, following each vendor's recommended starting model, not evidence that earlier models were withdrawn. Cross-host tiers pair by role and per-token price: `gpt-6-sol` ↔ `claude-sonnet-5`, -`gpt-6-astra` ↔ `claude-fable-5-1`, `gpt-6-luna` ↔ Haiku. Withdrawn ids are handled by the retirement +`gpt-6-astra` ↔ `claude-opus-5-5` (reasoning; Astra also pairs with `claude-fable-5-1`), +`gpt-6-luna` ↔ Haiku. See [ADR-0006](0006-primary-host-and-ambidextrous-mirroring.md) for tier parity. Withdrawn ids are handled by the retirement mechanism in [ADR-0003](0003-auto-seed-dual-host-provenance.md), not by editing this table alone. ## Consequences diff --git a/docs/adr/0006-primary-host-and-ambidextrous-mirroring.md b/docs/adr/0006-primary-host-and-ambidextrous-mirroring.md index ad7451ef..58c56f90 100644 --- a/docs/adr/0006-primary-host-and-ambidextrous-mirroring.md +++ b/docs/adr/0006-primary-host-and-ambidextrous-mirroring.md @@ -2,9 +2,12 @@ - **Status:** Amended by [ADR-0020](0020-ga-stable-surfaces.md) - **Date:** 2026-07-23 -- **Updated:** 2026-07-30 -- **Update note:** Preserved primary-host mirroring while removing the compatibility executor and - moving persisted intent to `routing.primaryHost`. +- **Updated:** 2026-09-23 +- **Update note:** Tier parity: both hosts now have a `reasoning` tier (`claude-opus-5-5` ↔ + `gpt-6-astra`), so a Codex-driven seed gets reasoning-tier models and an escalation that steps up a + tier, as a Claude-driven seed does. A catalog entry may `pairs` with an extra tier. +- **Earlier update (2026-07-30):** Preserved primary-host mirroring while removing the compatibility + executor and moving persisted intent to `routing.primaryHost`. - **Deciders:** agentic-kit maintainers > **GA amendment:** primary-host mirroring remains. The old command spelling, adapter dependency, @@ -51,9 +54,14 @@ orchestrator stays symmetric underneath. - Codex-primary is expressible in one flag at setup or pick time, with the whole experience (routing, escalation, status severity, dashboard indicator) following symmetrically. - The default (claude-primary) is unchanged, so existing repos see no difference. -- The mirror is coarse where host model tiers don't line up (claude tiers ≠ codex tiers): a - swapped route falls back to the counterpart host's recommended model. Users tune per activity - with `--route`. +- The mirror pairs models by tier. Both hosts carry `reasoning`, `balanced` and `fast` tiers + (`claude-opus-5-5` ↔ `gpt-6-astra`, `claude-sonnet-5` ↔ `gpt-6-sol`, Haiku ↔ `gpt-6-luna`), so + reasoning work and escalation land on the same tier whichever host drives. A model with no + same-tier twin uses an explicit `pairs` entry (`claude-fable-5-1` → `gpt-6-astra`), else the + counterpart host's recommended model. Users tune per activity with `--route`. +- Tier parity is not price parity: `gpt-6-astra` costs more per token than `claude-opus-5-5` + ($10/$50 vs $4/$20). Codex-driven reasoning therefore costs more at API rates, and uses more + Codex plan allowance, than Claude-driven reasoning. - `status`/dashboard must read `primaryHost` to render the correct severity + primary marker (done: `status.mjs` host rows, `dashboard-server.mjs` routing matrix). diff --git a/src/lib/routing.mjs b/src/lib/routing.mjs index 3b3a8b9a..5beec44a 100644 --- a/src/lib/routing.mjs +++ b/src/lib/routing.mjs @@ -81,8 +81,10 @@ export const MODEL_CATALOG_VERIFIED = '2026-09-23'; export const COST_AXIS_NOTE = 'per-token price ≠ per-task cost — compare total tokens, cache use and agentic turns on representative tasks'; // Tier names are the pairing key for swapHostModel(): a codex tier only mirrors // to a claude model (and back) when BOTH catalogs use the same tier string. -// Keep `flagship`/`balanced`/`fast` spelled identically on both hosts — -// renaming one side silently degrades every mirrored route to cat[0]. +// Keep `reasoning`/`balanced`/`fast` spelled identically on both hosts — +// renaming one side silently degrades every mirrored route to cat[0]. A model +// may also list `pairs`: extra tiers it answers for when the other host has no +// model of that exact tier (Astra stands in for claude's `flagship`). export const MODEL_CATALOG = { claude: [ // Opus 5.5 (released 2026-09-22) is the reasoning preset: Anthropic's @@ -105,12 +107,17 @@ export const MODEL_CATALOG = { // catalog ranks these first and labels every GPT-5.6 model "Older", and // OpenAI's Codex docs make Sol the default preset // (learn.chatgpt.com/docs/models, verified 2026-09-23). Tier pairing with the - // claude catalog follows role and per-token price: Sol ↔ Sonnet 5 ($2/$10 - // both), Astra ↔ Fable 5.1 ($10/$50 both), Luna ↔ Haiku. Sol is first, so a - // claude tier with no codex twin (reasoning) mirrors to the workhorse. + // claude catalog follows role: Sol ↔ Sonnet 5 (balanced, $2/$10 both), + // Luna ↔ Haiku (fast), and Astra ↔ Opus 5.5 (reasoning). Astra is the + // reasoning tier so a Codex-driven seed gets what a Claude-driven one does: + // reasoning work on the host's reasoning model and an escalation that steps + // UP a tier (Sonnet → Astra), not sideways to an equal-priced workhorse. + // Astra is priced above Opus 5.5 ($10/$50 vs $4/$20); OpenAI positions it for + // the hardest end-to-end work, and Anthropic reports Opus 5.5 at Fable 5.1 + // level, which is Astra's class. It also answers for claude's `flagship`. { id: 'gpt-6-sol', tier: 'balanced', note: 'workhorse preset for coding, testing and everyday work' }, { id: 'gpt-6-luna', tier: 'fast', note: 'fast-tier preset for mechanical work, documentation and packaging' }, - { id: 'gpt-6-astra', tier: 'flagship', note: 'frontier preset for the hardest end-to-end work; compare per-task cost before selection' }, + { id: 'gpt-6-astra', tier: 'reasoning', pairs: ['flagship'], note: 'reasoning preset for the hardest end-to-end work; compare per-task cost before selection' }, // Still served, retained for user pins. Codex labels them "Older". { id: 'gpt-5.6-sol', tier: 'prior', note: 'prior coding preset retained for user pins; compare measured per-task results before switching' }, { id: 'gpt-5.6-terra', tier: 'prior', note: 'prior balanced preset retained for user pins' }, @@ -224,14 +231,14 @@ const MODEL_TIER = Object.fromEntries( HOSTS.flatMap((h) => (MODEL_CATALOG[h] ?? []).map((m) => [m.id, m.tier])), ); -/** The opposite host's best model when swapping. Tier names differ between claude - * and codex, so an exact-tier match is best-effort; else fall back to that host's - * first (recommended) model. Pure. */ +/** The opposite host's best model when swapping: the model of the same tier, + * else one that `pairs` with that tier, else that host's first (recommended) + * model. Pure. */ export function swapHostModel(host, model) { const other = host === 'claude' ? 'codex' : 'claude'; const cat = MODEL_CATALOG[other] ?? []; const tier = MODEL_TIER[model]; - const pick = (tier && cat.find((m) => m.tier === tier)) || cat[0]; + const pick = (tier && (cat.find((m) => m.tier === tier) || cat.find((m) => m.pairs?.includes(tier)))) || cat[0]; return { host: other, model: pick?.id }; } diff --git a/tests/kit/routing-primary.test.mjs b/tests/kit/routing-primary.test.mjs index 7b48933a..5385331f 100644 --- a/tests/kit/routing-primary.test.mjs +++ b/tests/kit/routing-primary.test.mjs @@ -38,6 +38,40 @@ test('seedActivityRoutes with codex primary mirrors host assignments', () => { assert.equal(policy.implementation.host, 'claude'); // claude becomes the alternate }); +// Tier parity: a Codex-driven seed must get the same tier benefits a +// Claude-driven one does — reasoning work on each host's reasoning-tier model, +// and an escalation that steps UP a tier rather than sideways. +const REASONING_ACTIVITIES = ['architecture', 'design', 'security-analysis', 'debugging']; + +test('each host leads reasoning work on its own reasoning-tier model', () => { + const claudeLed = seedActivityRoutes({ hosts: ['claude', 'codex'] }); + const codexLed = seedActivityRoutes({ hosts: ['claude', 'codex'], primary: 'codex' }); + for (const act of REASONING_ACTIVITIES) { + assert.deepEqual([claudeLed[act].host, claudeLed[act].model], ['claude', 'claude-opus-5-5'], act); + assert.deepEqual([codexLed[act].host, codexLed[act].model], ['codex', 'gpt-6-astra'], act); + } +}); + +test('escalation steps up to the reasoning tier whichever host drives', () => { + const claudeLed = seedActivityRoutes({ hosts: ['claude', 'codex'] }); + const codexLed = seedActivityRoutes({ hosts: ['claude', 'codex'], primary: 'codex' }); + for (const act of ['implementation', 'testing']) { + assert.deepEqual(claudeLed[act].escalation, [{ host: 'claude', model: 'claude-opus-5-5' }], act); + assert.equal(codexLed[act].model, 'claude-sonnet-5', act); + assert.deepEqual(codexLed[act].escalation, [{ host: 'codex', model: 'gpt-6-astra' }], act); + } +}); + +test('tier pairs mirror both ways, and flagship pins reach Astra through an explicit pair', () => { + assert.equal(swapHostModel('claude', 'claude-opus-5-5').model, 'gpt-6-astra'); + assert.equal(swapHostModel('codex', 'gpt-6-astra').model, 'claude-opus-5-5'); + assert.equal(swapHostModel('claude', 'claude-fable-5-1').model, 'gpt-6-astra'); + assert.equal(swapHostModel('claude', 'claude-sonnet-5').model, 'gpt-6-sol'); + assert.equal(swapHostModel('codex', 'gpt-6-sol').model, 'claude-sonnet-5'); + assert.equal(swapHostModel('claude', 'claude-haiku-4-5-20251001').model, 'gpt-6-luna'); + assert.equal(swapHostModel('codex', 'gpt-6-luna').model, 'claude-haiku-4-5-20251001'); +}); + test('seedActivityRoutes stamps every seeded entry with provenance:seeded', () => { const policy = seedActivityRoutes({ hosts: ['claude', 'codex'], primary: 'codex' }); assert.ok(Object.values(policy).every((r) => r.provenance === 'seeded'));