Skip to content

ai-openrouter: function-tool converter strips cacheControl, so tool-definition prompt caching never reaches the wire #822

Description

@sbroms2

TanStack AI version

@tanstack/ai-openrouter 0.14.2 (with @tanstack/ai 0.34.0)

Framework/Library version

Node.js v24.15.0 — server-side, no UI framework (the bug is in the adapter's tool serialization, so it's framework-agnostic)

Describe the bug and the steps to reproduce it

Also relevant: @openrouter/sdk 0.12.35 (the OpenRouter SDK the adapter serializes through).

Summary

When routing Anthropic models through @tanstack/ai-openrouter, a cacheControl marker placed on a tool's metadata is dropped, so Anthropic prompt caching of tool definitions is impossible over OpenRouter. The sibling adapter @tanstack/ai-anthropic already forwards this (via convertCustomToolToAdapterFormat), so the two adapters are inconsistent.

Why it matters

A common cost optimization is to put a large, identical instruction/context block in a leading tool marked cache_control: ephemeral, so it caches across requests (Anthropic caches the toolssystemmessages prefix in order; a differing per-request schema can sit after the cached leading tool). Over OpenRouter this is currently impossible because the marker never reaches the wire.

Root cause

convertFunctionToolToAdapterFormat (packages/ai-openrouter/src/tools/function-tool.ts) builds { type, function } and does not read tool.metadata.cacheControl:

return {
  type: 'function',
  function: { name: tool.name, description: tool.description, parameters: inputSchema },
}

The OpenRouter SDK already supports the fieldChatFunctionToolFunction accepts cacheControl (camelCase) and remaps it to cache_control on the wire. The catch is its outbound Zod schema strips unrecognized keys, so a snake_case cache_control is dropped; the field must be the camelCase cacheControl.

Reproduction

Minimal runnable repro (no API key): see the linked example — npm install && node repro.mjs. Output shows cache_control absent on the wire (steps 1–2) and present once forwarded (step 3).

A function tool carrying metadata.cacheControl, replayed through the SDK's ChatRequest$outboundSchema (the same path the adapter uses), serializes without any cache_control field. With the one-line forward in the converter it serializes with cache_control: { type: 'ephemeral' }.

End-to-end through TanStack → OpenRouter → Anthropic (Sonnet 4.5), with the fix applied:

Call cache_write cache_read
1 (writes cache) 3111 0
2 (identical leading tool) 0 3111

Without the fix, both calls show cache_write: 0, cache_read: 0 — no caching occurs.

Suggested fix

Forward tool.metadata.cacheControl as a cacheControl field on the returned function tool, mirroring @tanstack/ai-anthropic's convertCustomToolToAdapterFormat. Additive and non-breaking (only present when supplied).

I have a PR ready with the fix + a wire-format test (replaying through ChatRequest$outboundSchema, matching the existing web-tools-wire-format.test.ts pattern) + a changeset.

Your Minimal, Reproducible Example - (Sandbox Highly Recommended)

https://stackblitz.com/github//tanstack-ai-openrouter-cachecontrol-repro

Screenshots or Videos (Optional)

No response

Do you intend to try to help solve this bug with your own PR?

Yes, I am also opening a PR that solves the problem along side this issue

Terms & Code of Conduct

  • I agree to follow this project's Code of Conduct
  • I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions