Repository navigation
Conversation
…rites separately, flag unknown models as unpriced. ## Why An audit of OpenLaw Benchmarks runs (@openlaw-1) found `claude-opus-5` and `claude-sonnet-5` billed at the Sonnet 4.x card (3 / 0.30 / 3.75 / 15 per MTok). Cause: aieo has no built-in per-model price for Anthropic, only the bifrost datasheet. When the datasheet is not loaded (STRUT_MODEL_CATALOG=off, a failed fetch, or a host that never calls startModelCatalog) or lacks a model, every Claude model falls back to `TOKEN_PRICING.anthropic`. ## What changed (src/aieo/provider.ts, src/pricing.ts, src/steps/core/agent.ts, src/steps/core/llm.ts) - **Built-in `ANTHROPIC_MODELS` table** with verified list prices (below). Lookup order for anthropic: datasheet (what the LLM gateway bills from), then this table. - **Id normalisation** (`normalizeAnthropicModelId`): `anthropic/claude-opus-5`, `claude-opus-5.5` (dotted), `claude-haiku-4-5-20251001` / `@20251001` all resolve to the bare entry. `getProviderForModel` maps a bare `claude-*` id to anthropic (below the LLM_PROVIDER check, like `gpt-` / `gemini-`). - **Unknown models are unpriced**: for anthropic, openai, google, xai, openrouter, a named model that neither the table nor the datasheet prices costs 0, logs one `[aieo] no price for <provider>/<model>: cost reported as 0 (unpriced)` warning, and the agent / llm step output carries `costUnpriced: true`. New `modelPriceSource(provider, model)` → table | catalog | provider-default | unpriced. Unchanged: the provider flat rate when no model is named, and the maple / numes / tokenfactory "dearest listed rate" ceiling for unlisted gateway models (a deliberate never-under-report policy, now commented). The explicit tokenfactory $0 entries (glm-5.3, kimi-k3.1-pro, …) are untouched. - **1-hour cache writes**: the agent step already supports `cacheTtl: "1h"`, but usage had one write count priced at the 5-minute rate. `TokenUsage.cacheWrite1hTokens` (a subset of cacheWriteTokens) is read from Anthropic's raw `cache_creation.ephemeral_1h_input_tokens` (usage.raw or providerMetadata.anthropic.usage). If there is no 5m/1h split and the step ran with cacheTtl 1h on anthropic, all its writes count as 1h. Priced at `cacheWrite1hPrice` (table, or the datasheet's `cache_creation_input_token_cost_above_1hr`). - **Haiku 5.5 long-context tier**: a call whose prompt (input + cache read + cache write) is over 100K tokens is priced entirely at the higher tier. ## Prices (USD / MTok: input / cache read / 5m write / 1h write / output) Sources: https://platform.claude.com/docs/en/about-claude/pricing and https://claude.com/pricing, both read 2026-11-11. The bifrost datasheet (https://getbifrost.ai/datasheet, same day) holds the same figures. | Model | Before (fallback when no datasheet) | After | |---|---|---| | claude-opus-5 | 3 / 0.30 / 3.75 / 3.75 / 15 | 5 / 0.50 / 6.25 / 10 / 25 | | claude-sonnet-5 | 3 / 0.30 / 3.75 / 3.75 / 15 | 2 / 0.20 / 2.50 / 4 / 10 | | claude-opus-5-5 | 3 / 0.30 / 3.75 / 3.75 / 15 | 4 / 0.20 / 5 / 8 / 20 | | claude-sonnet-5-5 | 3 / 0.30 / 3.75 / 3.75 / 15 | 2 / 0.10 / 2.50 / 4 / 10 | | claude-haiku-5-5 (≤100K / >100K prompt) | 3 / 0.30 / 3.75 / 3.75 / 15 | 0.10 / 0.01 / 0.125 / 0.20 / 0.50 · 0.50 / 0.05 / 0.625 / 1 / 2.50 | | claude-haiku-4-5 | 3 / 0.30 / 3.75 / 3.75 / 15 | 1 / 0.10 / 1.25 / 2 / 5 | | claude-opus-4-8, 4-7, 4-6, 4-5 | 3 / 0.30 / 3.75 / 3.75 / 15 | 5 / 0.50 / 6.25 / 10 / 25 | | claude-opus-4-1, claude-opus-4 | 3 / 0.30 / 3.75 / 3.75 / 15 | 15 / 1.50 / 18.75 / 30 / 75 | | claude-sonnet-4-6, 4-5, 4 | 3 / 0.30 / 3.75 / 3.75 / 15 | 3 / 0.30 / 3.75 / 6 / 15 (only the 1h write changes) | | claude-fable-5-2, claude-mythos-5-2 | 3 / 0.30 / 3.75 / 3.75 / 15 | 8 / 0.20 / 10 / 16 / 40 | | claude-fable-5-1, claude-mythos-5-1 | 3 / 0.30 / 3.75 / 3.75 / 15 | 10 / 0.25 / 12.50 / 20 / 50 | | claude-fable-5, claude-mythos-5 | 3 / 0.30 / 3.75 / 3.75 / 15 | 10 / 1 / 12.50 / 20 / 50 | | unknown claude-* | 3 / 0.30 / 3.75 / 3.75 / 15 | unpriced: 0 + warning + costUnpriced | Audit figures checked: Sonnet 5 (2 / 0.20 / 2.50 / 10) and Opus 5 (5 / 0.50 / 6.25 / 25) match the official pages exactly, and the 0.05x cache read on Opus 5.5 / Sonnet 5.5 is confirmed. Opus 5.5 is 4 / 20, not Opus 5's 5 / 25. Not modelled: Batch (-50%), fast mode, US-only inference_geo (1.1x). Other providers' per-model prices come from the datasheet as before. Their flat rates (openai, google, xai, openrouter) now apply only when no model is named, and their values are unchanged. ## Effect on run 1794343520308 (Opus 5 reasoning + Sonnet 5 standard) Every Opus 5 price is exactly 5/3 of the fallback card and every Sonnet 5 price exactly 2/3, so the billed figures rescale exactly (assuming 5-minute writes, the agent default): - Opus 5 steps: $18.87 → $31.45 - Sonnet 5 steps: $11.47 → $7.65 - Metered agents: **$30.34 → $39.10** (+$8.76, +28.9%) ## Tests New src/aieo/pricing-table.test.ts: table prices with no datasheet, prefix / dotted / dated ids, `claude-*` provider inference, unpriced behaviour (0, one warning, source), tokenfactory $0 kept, gateway ceiling kept, datasheet precedence, 1h write pricing and datasheet parsing, the Haiku 5.5 tier. Also pricing.test.ts (ephemeral_1h_input_tokens), llm.test.ts and agent.test.ts (costUnpriced). catalog.test.ts, runner.test.ts and job-threads.test.ts were updated for the new behaviour and the new usage field. `tsc --noEmit` is clean. The aieo, pricing, llm, runner, store, compaction, evaluate, job-threads and graph tests pass. agent.test.ts has 3 bash-process (ESRCH) failures that also fail on the base branch in the pod.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Price Claude models from Anthropic's list prices, bill 1-hour cache writes separately, flag unknown models as unpriced.
Why
An audit of OpenLaw Benchmarks runs (@openlaw-1) found
claude-opus-5andclaude-sonnet-5billed at the Sonnet 4.x card (3 / 0.30 / 3.75 / 15 per MTok). Cause: aieo has no built-in per-model price for Anthropic, only the bifrost datasheet. When the datasheet is not loaded (STRUT_MODEL_CATALOG=off, a failed fetch, or a host that never calls startModelCatalog) or lacks a model, every Claude model falls back toTOKEN_PRICING.anthropic.What changed (src/aieo/provider.ts, src/pricing.ts, src/steps/core/agent.ts, src/steps/core/llm.ts)
ANTHROPIC_MODELStable with verified list prices (below). Lookup order for anthropic: datasheet (what the LLM gateway bills from), then this table.normalizeAnthropicModelId):anthropic/claude-opus-5,claude-opus-5.5(dotted),claude-haiku-4-5-20251001/@20251001all resolve to the bare entry.getProviderForModelmaps a bareclaude-*id to anthropic (below the LLM_PROVIDER check, likegpt-/gemini-).[aieo] no price for <provider>/<model>: cost reported as 0 (unpriced)warning, and the agent / llm step output carriescostUnpriced: true. NewmodelPriceSource(provider, model)→ table | catalog | provider-default | unpriced. Unchanged: the provider flat rate when no model is named, and the maple / numes / tokenfactory "dearest listed rate" ceiling for unlisted gateway models (a deliberate never-under-report policy, now commented). The explicit tokenfactory $0 entries (glm-5.3, kimi-k3.1-pro, …) are untouched.cacheTtl: "1h", but usage had one write count priced at the 5-minute rate.TokenUsage.cacheWrite1hTokens(a subset of cacheWriteTokens) is read from Anthropic's rawcache_creation.ephemeral_1h_input_tokens(usage.raw or providerMetadata.anthropic.usage). If there is no 5m/1h split and the step ran with cacheTtl 1h on anthropic, all its writes count as 1h. Priced atcacheWrite1hPrice(table, or the datasheet'scache_creation_input_token_cost_above_1hr).Prices (USD / MTok: input / cache read / 5m write / 1h write / output)
Sources: https://platform.claude.com/docs/en/about-claude/pricing and https://claude.com/pricing, both read 2026-11-11. The bifrost datasheet (https://getbifrost.ai/datasheet, same day) holds the same figures.
Audit figures checked: Sonnet 5 (2 / 0.20 / 2.50 / 10) and Opus 5 (5 / 0.50 / 6.25 / 25) match the official pages exactly, and the 0.05x cache read on Opus 5.5 / Sonnet 5.5 is confirmed. Opus 5.5 is 4 / 20, not Opus 5's 5 / 25.
Not modelled: Batch (-50%), fast mode, US-only inference_geo (1.1x). Other providers' per-model prices come from the datasheet as before. Their flat rates (openai, google, xai, openrouter) now apply only when no model is named, and their values are unchanged.
Effect on run 1794343520308 (Opus 5 reasoning + Sonnet 5 standard)
Every Opus 5 price is exactly 5/3 of the fallback card and every Sonnet 5 price exactly 2/3, so the billed figures rescale exactly (assuming 5-minute writes, the agent default):
Tests
New src/aieo/pricing-table.test.ts: table prices with no datasheet, prefix / dotted / dated ids,
claude-*provider inference, unpriced behaviour (0, one warning, source), tokenfactory $0 kept, gateway ceiling kept, datasheet precedence, 1h write pricing and datasheet parsing, the Haiku 5.5 tier. Also pricing.test.ts (ephemeral_1h_input_tokens), llm.test.ts and agent.test.ts (costUnpriced). catalog.test.ts, runner.test.ts and job-threads.test.ts were updated for the new behaviour and the new usage field.tsc --noEmitis clean. The aieo, pricing, llm, runner, store, compaction, evaluate, job-threads and graph tests pass. agent.test.ts has 3 bash-process (ESRCH) failures that also fail on the base branch in the pod.