fix: annthropic cache rate for fast mode (#5063)

[TejasGhatte]

Summary

Anthropic's fast mode (speed:"fast") was billing cache tokens at standard rates instead of fast rates. This PR introduces dedicated fast-mode cache pricing columns (cache_creation_input_token_cost_fast, cache_creation_input_token_cost_above_1hr_fast, cache_read_input_token_cost_fast) and fixes the streaming path to forward the speed field onto the response so that tier.isFast is correctly set for streamed requests.

Changes

• Streaming fix: ToBifrostResponsesStream now copies chunk.Usage.Speed onto the response when emitting usage from message_delta events. Previously, streamed fast-mode requests had tier.isFast = false, causing all cache and input/output tokens to bill at standard rates.
• Fast-mode cache pricing columns: Three new columns (cache_creation_input_token_cost_fast, cache_creation_input_token_cost_above_1hr_fast, cache_read_input_token_cost_fast) are added to TableModelPricing. When set, tieredCacheReadInputTokenRate, tieredCacheCreationInputTokenRate, and tieredCacheCreationInputAbove1hrTokenRate return the fast rate instead of the standard or tiered rate.
• Database migration: migrationAddFastModeCachePricingColumns adds the three columns with rollback support.
• Pricing sync: The new columns are included in pricingSyncUpdateColumns so they are written during catalog syncs.
• Overrides and catalog types: Options, Entry conversion helpers, and patchPricing are updated to carry the new fields through the override and catalog pipeline.
• UI: The custom pricing override sheet exposes the three new fast cache fields, and the PricingOverridePatch TypeScript type is updated to match.
• Tests: The existing TestComputeTextCost_FastMode_CacheBillsAtStandardRates test is replaced with TestComputeTextCost_FastMode_UsesFastCacheRates and two new tests — one verifying fallback to standard rates when fast columns are unset, and a regression test pinning the real-world Opus 4.8 miscalculation. A streaming test (TestFastMode_StreamingForwardsSpeed) verifies the message_delta path forwards speed=fast and preserves cache-creation token counts.

Type of change

• Bug fix
• Feature

Affected areas

• Core (Go)
• Providers/Integrations
• UI (React)

How to test

# Core/Transports
go test ./core/providers/anthropic/... ./framework/modelcatalog/datasheet/... ./framework/configstore/...

# UI
cd ui
pnpm i
pnpm build

1. Configure a fast-mode Anthropic model with cache_creation_input_token_cost_fast and cache_read_input_token_cost_fast set.
2. Send a streaming request with cache_control blocks and speed:"fast".
3. Verify the billed cost uses the fast cache rates, not the standard rates.
4. Verify that when the fast cache columns are absent, billing falls back to standard cache rates without error.

Breaking changes

• No

Security considerations

None. Changes are limited to pricing calculation logic, database schema, and UI display fields.

Checklist

• I added/updated tests where appropriate
• I verified builds succeed (Go and UI)

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[TejasGhatte]

• tests: openai cache coverage #5091
• fix: openai cache write pricings #5090
• fix: service tier in openai chat completion (#5073) #5089
• fix: inference geo cost on anthropic (#5072) #5088
• fix: annthropic cache rate for fast mode (#5063) #5087 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[greptile-apps]

Confidence Score: 4/5

The fast-mode streaming billing path has a contained issue in the final completed response.

• The new pricing fields appear wired through storage, overrides, sync, and the UI.
• The migration follows the existing nullable-column pattern.
• The completed Anthropic stream response can still miss Speed, so billing from that response can use the wrong tier.

core/providers/anthropic/responses.go

Important Files Changed

Filename	Overview
core/providers/anthropic/responses.go	Forwards streamed speed on usage chunks, but not on the completed stream response.
framework/configstore/migrations.go	Adds a rollbackable migration for nullable fast cache pricing columns.
framework/configstore/rdb.go	Adds the new fast cache pricing columns to pricing sync updates.
framework/configstore/tables/modelpricing.go	Adds persisted fields for Anthropic fast cache pricing.
framework/modelcatalog/datasheet/cost.go	Uses fast cache rates before falling back to existing standard cache tiers.
framework/modelcatalog/datasheet/overrides.go	Applies fast cache pricing values from override patches.
framework/modelcatalog/datasheet/types.go	Adds fast cache pricing fields to pricing options and table conversions.
ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx	Adds fast cache pricing fields to the custom pricing sheet.
ui/lib/types/governance.ts	Adds matching TypeScript fields for pricing override patches.

_{Reviews (1): Last reviewed commit: "fix: annthropic cache rate for fast mode..." | Re-trigger Greptile}

[akshaydeo]

Merge activity

• Jul 10, 10:32 AM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jul 10, 10:32 AM UTC: @akshaydeo merged this pull request with Graphite.