tests: openai cache coverage

[TejasGhatte]

Summary

Adds Cross-Cut Round 31 to the provider harness E2E test collection, covering OpenAI prompt caching across the new gpt-5.6 model family (sol, terra, luna) and legacy models (gpt-4o, gpt-4.1, gpt-5, etc.). The suite validates that prompt_cache_options, prompt_cache_breakpoint, and cache_write_tokens fields are correctly passed through and that cache write/read cycles produce the expected token accounting in both streaming and non-streaming modes.

Changes

• Added a stable per-run pcNonce collection variable (set once in the pre-request script) so that paired [write] and [read] requests within a run share the same cache key prefix, enabling cold-then-warm cache hit verification without cross-run collisions.
• Added a cachePrefix collection variable containing a large, stable, deliberately verbose prompt text that meets provider minimum token thresholds for prompt caching eligibility.
• Added Round 31 test group with four sub-groups:
  • 31.1 — gpt-5.6 new fields, non-streaming: validates prompt_cache_options (mode + TTL), prompt_cache_breakpoint, and cache_write_tokens > 0 on write, then cached_tokens > 0 on read, across native chat, native responses, drop-in chat, and drop-in responses endpoints for sol, terra, and luna variants.
  • 31.2 — gpt-5.6 new fields, streaming: same coverage as 31.1 but over SSE, additionally asserting text/event-stream content-type and that usage appears in the stream.
  • 31.3 — Legacy automatic caching, non-streaming: validates cached_tokens shape (numeric on write, > 0 on read) for gpt-5, gpt-5-mini, gpt-5-nano, and gpt-4.1 across native and drop-in endpoints.
  • 31.4 — Legacy automatic caching, streaming: same as 31.3 but over SSE for gpt-4.1-mini, gpt-4.1-nano, gpt-4o, and gpt-4o-mini.
• Each [read] request includes a ~1.5 s busy-wait pre-request script to allow OpenAI's cache to propagate before asserting a cache hit.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

Run the collection via Newman against a live Bifrost instance with valid provider keys:

newman run tests/e2e/api/collections/provider-harness.json \
  --env-var baseUrl=http://localhost:8080 \
  --env-var openaiKey=sk-... \
  --folder "Cross-Cut Round 31: OpenAI Prompt Caching"

To also exercise the compat path:

newman run tests/e2e/api/collections/provider-harness.json \
  --env-var baseUrl=http://localhost:8080 \
  --env-var openaiKey=sk-... \
  --env-var compat=true \
  --folder "Cross-Cut Round 31: OpenAI Prompt Caching"

Expected outcome: all [write] requests report cache_write_tokens > 0 and all [read] requests report cached_tokens > 0.

Screenshots/Recordings

N/A — no UI changes.

Breaking changes

• Yes
• No

Related issues

N/A

Security considerations

The cachePrefix collection variable contains only static, non-sensitive harness text. No secrets, PII, or auth material is introduced. API keys remain in environment variables and are not committed.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[coderabbitai]

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

💤 Files selected but had no reviewable changes (1)

• tests/e2e/api/collections/provider-harness.json

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 3e915ee2-d6ca-4408-bfad-7e461fe0a55f

📥 Commits

Reviewing files that changed from the base of the PR and between d65efa0 and ae8cd56.

📒 Files selected for processing (1)

• tests/e2e/api/collections/provider-harness.json

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 07-10-tests_openai_cache_coverage

---

_{Comment @coderabbitai help to get the list of available commands.}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

tejas ghatte seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[TejasGhatte]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• tests: openai cache coverage #5086 👈 (View in Graphite)
• fix: openai cache write pricings #5084
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[greptile-apps]

Confidence Score: 4/5

The new harness coverage has one contained Responses streaming test issue.

• The changed collection is limited to e2e prompt-cache coverage.
• Responses streaming tests can fail because they assert streamed usage without requesting it.
• No security-sensitive runtime code changed.

tests/e2e/api/collections/provider-harness.json

Important Files Changed

Filename	Overview
tests/e2e/api/collections/provider-harness.json	Adds Round 31 OpenAI prompt-cache harness coverage; the Responses streaming request bodies need streamed usage enabled for their usage-based assertions.

_{Reviews (1): Last reviewed commit: "tests: openai cache coverage" | Re-trigger Greptile}