Skip to content

fix(core): replay Claude reasoning signatures for the truefoundry provider - #1424

Open
ramantehlan wants to merge 8 commits into
mainfrom
sd/truefoundry-reasoning-signature
Open

ramantehlan wants to merge 8 commits into
mainfrom
sd/truefoundry-reasoning-signature

Conversation

@ramantehlan

@ramantehlan ramantehlan commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Reasoning sent through the TrueFoundry AI gateway was replayed unsigned, so Bedrock dropped the thinking block. The model lost its plan on every iteration, kept calling tools, and ran to iteration_limit with no answer.

Closes #1347

Root cause

Three gaps, two in @ai-sdk/openai-compatible and one of ours:

  1. Capture. The adapter reads delta.reasoning_content and nothing else. reasoning_signature appears zero times in the package, so the signature never reached the stream parts and applyReasoningSignature had no providerMetadata to copy from.
  2. Replay, ours. toAssistantModelMessage only built signature providerOptions for openai and anthropic. Even a perfect adapter would have attached nothing for truefoundry.
  3. Replay, adapter. It flattens every reasoning part into one reasoning_content string, which has no slot for a signature.

This has been broken since the truefoundry provider type was added in #483. It is invisible on Vertex and Microsoft Foundry, which accept unsigned reasoning; only Bedrock enforces it.

What the gateway actually sends

Traced delta-by-delta against a live gateway on claude-sonnet-5:

0-7   reasoning_content deltas   (streamed incrementally, unsigned)
8     reasoning_signature        (once, right after reasoning ends)
9-55  content deltas
56-57 tool_calls
58    thinking_blocks            (once, at the very end, consolidated:
                                  full thinking text + signature embedded)

The trailing thinking_blocks array is already exactly our ThinkingBlock shape, so this takes it wholesale rather than pairing a loose signature with an open block. That needs no ordering logic and keeps interleaved thinking intact.

Changes

  • New truefoundryReasoning.ts: a MetadataExtractor that pulls the gateway's consolidated thinking_blocks off raw chunks, plus the readers for getting them back out.
  • compatibleModel passes that extractor, for truefoundry only.
  • mapStreamToChunks reads the signed blocks off the finish-step part, where extracted metadata lands, and lets them supersede the unsigned blocks built from reasoning deltas.
  • toAssistantModelMessage replays them via the adapter's message-level openaiCompatible passthrough. Reasoning parts are skipped in that case, so the gateway is not handed the same reasoning twice, once unsigned.
  • Thinking-block parsing now goes through the existing ThinkingBlockSchema instead of hand-rolled in guards. Malformed entries are still skipped rather than failing the message.

No patch to @ai-sdk/openai-compatible. Both directions use supported hooks, and patches/ stays empty.

How was this tested?

  • New truefoundryReasoning.test.ts (8 tests) drives the extractor with chunk shapes captured from the live gateway: the trailing consolidated block, interleaved thinking, streams that never sign, malformed input, and the non-streamed body.
  • New round-trip suite in VercelAILLM.replay.test.ts (3 tests) asserts the exact outbound wire message, that the unsigned path still produces reasoning_content, and that a block signed by a different configured provider is not replayed.
  • Both halves were confirmed to fail independently when reverted.
  • pnpm test (606 passing), pnpm typecheck, pnpm lint:ci, pnpm format:check pass locally.

Not verified end to end. The dev gateway has no Bedrock route (claude-sonnet-5 goes to Vertex, claude-opus-4-8 to Microsoft Foundry), so confirming the loop is gone needs a Bedrock-backed Claude account. The unit tests pin the wire shape the gateway docs ask for.

Checklist

  • I have read the contributing guidelines
  • pnpm test, pnpm typecheck, pnpm lint:ci, and pnpm format:check pass locally
  • Tests added/updated where it makes sense
  • No hand-edits to generated code
  • Docs / .env.example updated if configuration or behavior changed (no config change)

Note

Medium Risk
Touches core LLM streaming, message replay, and multi-provider routing; mistakes could break tool loops or mishandle signatures across provider switches, though behavior is gated and heavily tested.

Overview
Fixes multi-turn agent loops on Bedrock-backed Claude through TrueFoundry and other OpenAI-compatible providers (truefoundry, custom, fireworks, zai, together) by preserving and replaying signed thinking_blocks instead of unsigned reasoning_content alone.

A new openAICompatibleReasoning layer plugs a MetadataExtractor into @ai-sdk/openai-compatible clients so consolidated gateway thinking_blocks from trailing stream/response chunks are captured. mapStreamToChunks prefers those signed blocks over deltas accumulated during streaming, and toAssistantModelMessage puts them back on the wire via message-level openaiCompatible.thinking_blocks when signatures or redacted blocks require whole-block replay—avoiding duplicate unsigned reasoning on the same turn.

ThinkingBlockUnionSchema is now a discriminated union (thinking vs redacted), and replay stays provider-bound when the session switches gateway type or configured provider name. New unit and cross-layer replay tests cover gateway wire shapes end to end.

Reviewed by Cursor Bugbot for commit 701e02e. Bugbot is set up for automated code reviews on this repo. Configure here.

…vider

Reasoning sent through the TrueFoundry AI gateway was replayed unsigned, so
Bedrock dropped the thinking block and the model lost its plan on every
iteration, looping until iteration_limit with no answer.

@ai-sdk/openai-compatible reads delta.reasoning_content and nothing else, so
the signature the gateway sends alongside it never reached the stream parts.
Even once captured it had nowhere to go: toAssistantModelMessage only built
signature providerOptions for openai and anthropic, and the adapter flattens
reasoning parts into a bare reasoning_content string.

Capture the gateway's consolidated thinking_blocks via a metadataExtractor and
replay them whole through the adapter's message-level passthrough. Both hooks
are supported, so no patch to the package is needed.

Fixes #1347

Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
@changeset-bot

changeset-bot Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 95ced07

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@truefoundry/trueforge-core Patch
@truefoundry/trueforge Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@veria-ai

veria-ai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

PR overview

No security issues were found in this pull request.

Reviews

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b9c3665. Configure here.

…ted ones

Claude returns redacted_thinking blocks interleaved with plain ones, and the
extractor parsed thinking_blocks as an array of plain blocks only. One redacted
entry failed the whole payload, so no metadata reached finish-step and replay
went back unsigned again — the exact failure this was meant to fix.

Parse with ThinkingBlockUnionSchema, which the stored and streamed message
schemas already use, and replay the whole ordered array. Redacted blocks now
survive replay too; they carry no text, so they are skipped only where a
reasoning part is being built.

Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
The signature capture and replay was gated to the `truefoundry` provider
type, but the gap is in `@ai-sdk/openai-compatible` itself, so every type
served by that adapter had it: `custom`, `fireworks`, `zai`, `together`.
`custom` is the one that matters, since it is how LiteLLM and OpenRouter
gateways fronting Claude are configured.

Replay stays capture-driven and `shouldAttachReasoningSignature` already
pins blocks to the provider that produced them, so a gateway that sends no
thinking blocks is unaffected.

Rename the module and its exports to drop the provider-specific naming, and
give the extracted metadata its own namespace so it cannot collide with the
adapter's own per-provider metadata.

Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
A session can change model between turns, so replay has to cope with
history signed by whoever answered last. A signature only means something
to the provider that issued it, and the gateway path carries blocks on the
message rather than on a reasoning part, so it needed its own coverage.

Covers gateway history replayed to anthropic/openai/google-gemini, native
history replayed to a gateway, and two gateways of the same type under
different configured names. Also pins the one case that does still replay:
same provider, different model.

Breaking the source guard fails five of these plus the two that already
existed; the google-gemini case is held by a different mechanism, since no
branch attaches a signature for it either way.

Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
@thesujai

Copy link
Copy Markdown
Contributor

After merge lets ensure following in devtest and then only close the ticket:

1. reasoning is tested both ways gateway to tf and tf to gateway
2. cross provider switch works
3. tool signatures work

- Rename hasReplayableSignature to needsWholeBlockReplay: a redacted block
  carries no signature, only an opaque payload, so the old name described
  one of the two cases it covers.
- Name the metadata namespace openaiCompatibleThinking after the adapter it
  belongs to rather than the product.
- Drop two defensive array copies. Zod returns a fresh array from safeParse,
  so spreading it again bought nothing.

Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
The extractor tests and the replay tests each hand-wrote the metadata key, so
the real extractor was never in the loop that produces the outbound message.
Drive it from raw gateway chunks instead, end to end.

Also cover a session switching between gateways of different types, which was
the only half of the source check with no test behind it.

Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reasoning is not replayed for the truefoundry provider (no reasoning_signature or thinking_blocks), so turns loop to iteration_limit

3 participants