Skip to content

[NA] [BE] fix: register fireworks_ai as a canonical provider so Fireworks model prices load#7554

Merged
andrescrz merged 1 commit into
comet-ml:mainfrom
Anuj7411:Anuj7411/fireworks-provider-registration
Jul 21, 2026
Merged

[NA] [BE] fix: register fireworks_ai as a canonical provider so Fireworks model prices load#7554
andrescrz merged 1 commit into
comet-ml:mainfrom
Anuj7411:Anuj7411/fireworks-provider-registration

Conversation

@Anuj7411

@Anuj7411 Anuj7411 commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Details

Fireworks AI's pricing entries are silently dropped by Opik today. model_prices_and_context_window.json carries roughly 260 non-zero-cost entries tagged litellm_provider: "fireworks_ai" (the accounts/fireworks/models/* catalog spanning the llama, qwen, deepseek, glm, kimi and minimax families), and none of them load because fireworks_ai is not registered in PROVIDERS_MAPPING; buildModelPrice returns null for any entry whose provider is not in that map, so every Fireworks span ends up with no cost. This follows the exact pattern of the earlier provider-registration fixes for azure (#7262), xai (#7433), deepseek (#7434) and perplexity (#7432).

  • Register fireworks_ai -> fireworks_ai in PROVIDERS_MAPPING so the entries load
  • Route fireworks_ai through textGenerationWithCacheCostOpenAI in PROVIDERS_CACHE_COST_CALCULATOR; Fireworks' LiteLLM cost calculator (litellm/llms/fireworks_ai/cost_calculator.py) delegates to generic_cost_per_token, so its usage payload uses the OpenAI shape (cached tokens under prompt_tokens_details.cached_tokens), and the eight cache-priced Fireworks models today (glm-4p7, glm-5p1, kimi-k2p5, minimax-m2p1) get the same cache-read discount OpenAI/Azure/xAI/DeepSeek receive
  • PROVIDERS_CACHE_COST_CALCULATOR was at the ten-pair ceiling of Map.of, so it moves to Map.ofEntries to take the eleventh entry

Change checklist

  • User facing
  • Documentation update

Issues

No ticket; discovered by scanning litellm_provider values in the bundled price map for providers with priced models that are absent from PROVIDERS_MAPPING.

AI-WATERMARK

AI-WATERMARK: yes

  • Tools: Claude Code
  • Model(s): Claude Opus 4.8
  • Scope: investigation of the drop path, the two-line mapping change, and the parameterized test
  • Human verification: model resolution and both cost figures checked against the bundled model_prices_and_context_window.json before submission

Testing

Added a parameterized test (calculateCostHandlesFireworksModels) covering both branches:

  • fireworks_ai/accounts/fireworks/models/llama-v3p1-70b-instruct (input 9e-7, output 9e-7, no cache) routes through textGenerationCost; 1000 prompt + 200 completion tokens = 0.00108
  • fireworks_ai/accounts/fireworks/models/kimi-k2p5 (input 6e-7, output 3e-6, cache_read 1e-7) routes through textGenerationWithCacheCostOpenAI; with 300 of 1000 prompt tokens cached, 700 * 6e-7 + 200 * 3e-6 + 300 * 1e-7 = 0.00105

Both cases fail against the current code (the models do not resolve without the mapping entry, so cost comes back as zero) and pass with the fix.

Documentation

No documentation changes needed; provider pricing is driven entirely by the bundled price map and the provider registration in CostService.

Follow-up

The same drop affects several other high-volume OpenAI-compatible inference providers that could be registered the same way if the team wants them: openrouter (~92 priced models), novita (~85), deepinfra (~67), together_ai (~36), databricks (~28) and moonshot (~22). Happy to open follow-ups for any of those; keeping this PR scoped to Fireworks alone.

…model prices load

Following the pattern of comet-ml#7262 (azure), comet-ml#7433 (xai), comet-ml#7434 (deepseek) and comet-ml#7432
(perplexity), Fireworks AI's ~260 non-zero-cost entries in
model_prices_and_context_window.json (the accounts/fireworks/models/* catalog:
llama, qwen, deepseek, glm, kimi, minimax families and more) currently drop at
load time because fireworks_ai is not in PROVIDERS_MAPPING, so buildModelPrice
returns null for every one of them and Fireworks spans never get a cost.

Add fireworks_ai -> fireworks_ai to PROVIDERS_MAPPING, and route it through
textGenerationWithCacheCostOpenAI in PROVIDERS_CACHE_COST_CALCULATOR. Fireworks'
cost calculator in LiteLLM (litellm/llms/fireworks_ai/cost_calculator.py)
delegates to generic_cost_per_token, so its usage payload uses the same OpenAI
shape (cached tokens flattened under prompt_tokens_details.cached_tokens); the 8
cache-priced Fireworks models today (glm-4p7, glm-5p1, kimi-k2p5, minimax-m2p1)
therefore get the same cache-read discount that OpenAI/Azure/xAI/DeepSeek do.

PROVIDERS_CACHE_COST_CALCULATOR was at the 10-pair ceiling of Map.of, so it moves
to Map.ofEntries to take the eleventh entry.

Parameterized test covers both branches: a Fireworks model with no cache rates
(plain textGenerationCost) and one with cache rates (cache-aware OpenAI route).
@Anuj7411
Anuj7411 requested a review from a team as a code owner July 21, 2026 15:15
@github-actions github-actions Bot added java Pull requests that update Java code Backend tests Including test files, or tests related like configuration. 🟢 size/S labels Jul 21, 2026
@Anuj7411 Anuj7411 changed the title [BE] fix: register fireworks_ai as a canonical provider so Fireworks model prices load [NA] [BE] fix: register fireworks_ai as a canonical provider so Fireworks model prices load Jul 21, 2026

@andrescrz andrescrz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Anuj7411

Thanks for the contribution! I reviewed the PR and everything looks good from my side. I've approved it, and all CI checks have passed successfully.

We'll merge it shortly, and the change will be included in the next Opik release.

Thanks again for contributing to Opik!

@andrescrz
andrescrz merged commit fa6b031 into comet-ml:main Jul 21, 2026
33 of 38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Backend java Pull requests that update Java code 🟢 size/S tests Including test files, or tests related like configuration.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants