tests: openai cache coverage

[TejasGhatte]

Summary

Adds Cross-Cut Round 31 to the provider harness E2E test suite, covering OpenAI prompt caching across the new gpt-5.6 model family (sol, terra, luna) and legacy models (gpt-4o, gpt-4.1, gpt-5, etc.). The suite validates that prompt_cache_options, prompt_cache_breakpoint, and cache_write_tokens fields are correctly passed through and that cold-write/warm-read cache cycles produce the expected token accounting in both streaming and non-streaming modes.

Changes

• Added a pcNonce collection variable initialized once per run in the pre-request script so that all write/read request pairs within a run share a stable, unique cache key prefix without colliding across runs.
• Added a cachePrefix collection variable containing a deliberately verbose, stable system prompt designed to exceed provider minimum token thresholds for prompt caching eligibility.
• Added Round 31 test group with four sub-groups:
  • 31.1 — gpt-5.6 (sol, terra, luna) non-streaming: tests prompt_cache_options + prompt_cache_breakpoint + cache_write_tokens on both /v1/chat/completions and /v1/responses endpoints via native and drop-in routes.
  • 31.2 — gpt-5.6 streaming: same coverage as 31.1 with SSE content-type assertions and streamed usage extraction.
  • 31.3 — Legacy models (gpt-5, gpt-5-mini, gpt-5-nano, gpt-4.1) non-streaming: validates cached_tokens shape and cache hit on automatic (implicit) caching without explicit breakpoints.
  • 31.4 — Legacy models (gpt-4.1-mini, gpt-4.1-nano, gpt-4o, gpt-4o-mini) streaming: same as 31.3 but over SSE.
• Each test case follows a write→read pattern with a ~1.5 s busy-wait before read requests to allow cache propagation.
• Write assertions check cache_write_tokens > 0; read assertions check cached_tokens > 0. Responses API write tests additionally assert that prompt_cache_options is echoed back in the response.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

Run the collection via Newman against a live Bifrost instance with valid OpenAI credentials:

newman run tests/e2e/api/collections/provider-harness.json \
  --env-var baseUrl=<your-bifrost-url> \
  --env-var openaiKey=<your-openai-key> \
  --folder "Cross-Cut Round 31: OpenAI Prompt Caching"

Expected outcome: all write requests report cache_write_tokens > 0 on cold cache, and all paired read requests report cached_tokens > 0 confirming a cache hit. Run the suite twice with different pcNonce values to confirm isolation between runs.

Screenshots/Recordings

N/A — no UI changes.

Breaking changes

• No

Related issues

N/A

Security considerations

The cachePrefix value is a static, non-sensitive string used solely to pad prompts to caching-eligible token lengths. No secrets, PII, or auth material is introduced.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[TejasGhatte]

• tests: openai cache coverage #5091 👈 (View in Graphite)
• fix: openai cache write pricings #5090
• fix: service tier in openai chat completion (#5073) #5089
• fix: inference geo cost on anthropic (#5072) #5088
• fix: annthropic cache rate for fast mode (#5063) #5087
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

tejas ghatte seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[coderabbitai]

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

💤 Files selected but had no reviewable changes (1)

• tests/e2e/api/collections/provider-harness.json

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: acf848d5-4d6d-451b-bdd6-e28591048046

📥 Commits

Reviewing files that changed from the base of the PR and between b610755 and 7176f83.

📒 Files selected for processing (1)

• tests/e2e/api/collections/provider-harness.json

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 07-10-cp-openai-cache-tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Confidence Score: 4/5

This is close, but the Responses streaming cases should be fixed before merging.

• Responses streaming requests assert usage-derived cache fields without ensuring usage is emitted.
• This can make the new cache tests fail before they check the cache behavior they were added to cover.

tests/e2e/api/collections/provider-harness.json

Important Files Changed

Filename	Overview
tests/e2e/api/collections/provider-harness.json	Adds Round 31 prompt-caching coverage, but Responses streaming cache requests still omit the usage opt-in that their assertions depend on.

_{Reviews (4): Last reviewed commit: "tests: openai cache coverage" | Re-trigger Greptile}

[akshaydeo]

Merge activity

• Jul 10, 10:32 AM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jul 10, 10:40 AM UTC: @akshaydeo merged this pull request with Graphite.

The base branch was changed.