Skip to content

feat: enhance token usage resolution for OpenAI and Anthropic compati…#223

Open
AlapinEnjoyer wants to merge 1 commit into
alibaba:mainfrom
Val-Dennis:feat/recognize-openai-cached-tokens
Open

feat: enhance token usage resolution for OpenAI and Anthropic compati…#223
AlapinEnjoyer wants to merge 1 commit into
alibaba:mainfrom
Val-Dennis:feat/recognize-openai-cached-tokens

Conversation

@AlapinEnjoyer

Copy link
Copy Markdown
Contributor

Description

Cached token usage was not getting tracked for the OpenAI provider, the MR adds a way to recognize the common OpenAI-compatible cached-token fields for custom providers

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Refactoring (no functional changes)
  • Documentation update
  • CI / Build / Tooling

How Has This Been Tested?

  • make test passes locally
  • Manual testing (describe below)

Tested with a custom provider run

Checklist

  • My code follows the project's coding style (go fmt, go vet)
  • I have performed a self-review of my code
  • I have added tests that prove my fix is effective or my feature works
  • New and existing unit tests pass locally with my changes
  • I have updated the documentation accordingly (if applicable)
  • I have signed the CLA

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 OpenCodeReview found 1 issue(s) in this PR.

  • ✅ 1 posted as inline comment(s)
  • 📝 0 posted as summary

Comment on lines +45 to +46
"usage.prompt_tokens_details.cache_creation_tokens",
"data.usage.prompt_tokens_details.cache_creation_tokens",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor maintainability issue: These two new entries lack comments describing which providers or API formats use cache_creation_tokens under prompt_tokens_details. All other entries in these path lists have explanatory comments. Adding a comment (e.g., // OpenAI-compatible providers or similar) would keep the style consistent and help future maintainers understand when this path applies.

Suggestion:

Suggested change
"usage.prompt_tokens_details.cache_creation_tokens",
"data.usage.prompt_tokens_details.cache_creation_tokens",
"usage.prompt_tokens_details.cache_creation_tokens", // OpenAI-compatible providers
"data.usage.prompt_tokens_details.cache_creation_tokens", // wrapped OpenAI-compatible providers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant