Repository navigation
fix(core): replay Claude reasoning signatures for the truefoundry provider - #1424
Open
ramantehlan wants to merge 8 commits into
Open
ramantehlan wants to merge 8 commits into
ramantehlan wants to merge 8 commits into
Conversation
…vider Reasoning sent through the TrueFoundry AI gateway was replayed unsigned, so Bedrock dropped the thinking block and the model lost its plan on every iteration, looping until iteration_limit with no answer. @ai-sdk/openai-compatible reads delta.reasoning_content and nothing else, so the signature the gateway sends alongside it never reached the stream parts. Even once captured it had nowhere to go: toAssistantModelMessage only built signature providerOptions for openai and anthropic, and the adapter flattens reasoning parts into a bare reasoning_content string. Capture the gateway's consolidated thinking_blocks via a metadataExtractor and replay them whole through the adapter's message-level passthrough. Both hooks are supported, so no patch to the package is needed. Fixes #1347 Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
ramantehlan
requested review from
chiragjn,
debajyoti-truefoundry,
heerambavi1998 and
thesujai
as code owners
November 18, 2026 08:09
🦋 Changeset detectedLatest commit: 95ced07 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b9c3665. Configure here.
…ted ones Claude returns redacted_thinking blocks interleaved with plain ones, and the extractor parsed thinking_blocks as an array of plain blocks only. One redacted entry failed the whole payload, so no metadata reached finish-step and replay went back unsigned again — the exact failure this was meant to fix. Parse with ThinkingBlockUnionSchema, which the stored and streamed message schemas already use, and replay the whole ordered array. Redacted blocks now survive replay too; they carry no text, so they are skipped only where a reasoning part is being built. Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
The signature capture and replay was gated to the `truefoundry` provider type, but the gap is in `@ai-sdk/openai-compatible` itself, so every type served by that adapter had it: `custom`, `fireworks`, `zai`, `together`. `custom` is the one that matters, since it is how LiteLLM and OpenRouter gateways fronting Claude are configured. Replay stays capture-driven and `shouldAttachReasoningSignature` already pins blocks to the provider that produced them, so a gateway that sends no thinking blocks is unaffected. Rename the module and its exports to drop the provider-specific naming, and give the extracted metadata its own namespace so it cannot collide with the adapter's own per-provider metadata. Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
…re' into sd/truefoundry-reasoning-signature
A session can change model between turns, so replay has to cope with history signed by whoever answered last. A signature only means something to the provider that issued it, and the gateway path carries blocks on the message rather than on a reasoning part, so it needed its own coverage. Covers gateway history replayed to anthropic/openai/google-gemini, native history replayed to a gateway, and two gateways of the same type under different configured names. Also pins the one case that does still replay: same provider, different model. Breaking the source guard fails five of these plus the two that already existed; the google-gemini case is held by a different mechanism, since no branch attaches a signature for it either way. Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
thesujai
reviewed
Nov 19, 2026
thesujai
reviewed
Nov 19, 2026
thesujai
reviewed
Nov 19, 2026
thesujai
reviewed
Nov 19, 2026
thesujai
reviewed
Nov 19, 2026
thesujai
reviewed
Nov 19, 2026
thesujai
approved these changes
Nov 19, 2026
Contributor
|
After merge lets ensure following in devtest and then only close the ticket: |
- Rename hasReplayableSignature to needsWholeBlockReplay: a redacted block carries no signature, only an opaque payload, so the old name described one of the two cases it covers. - Name the metadata namespace openaiCompatibleThinking after the adapter it belongs to rather than the product. - Drop two defensive array copies. Zod returns a fresh array from safeParse, so spreading it again bought nothing. Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
The extractor tests and the replay tests each hand-wrote the metadata key, so the real extractor was never in the loop that produces the outbound message. Drive it from raw gateway chunks instead, end to end. Also cover a session switching between gateways of different types, which was the only half of the source check with no test behind it. Signed-off-by: Raman Tehlan <ramantehlan@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Reasoning sent through the TrueFoundry AI gateway was replayed unsigned, so Bedrock dropped the thinking block. The model lost its plan on every iteration, kept calling tools, and ran to
iteration_limitwith no answer.Closes #1347
Root cause
Three gaps, two in
@ai-sdk/openai-compatibleand one of ours:delta.reasoning_contentand nothing else.reasoning_signatureappears zero times in the package, so the signature never reached the stream parts andapplyReasoningSignaturehad noproviderMetadatato copy from.toAssistantModelMessageonly built signatureproviderOptionsforopenaiandanthropic. Even a perfect adapter would have attached nothing fortruefoundry.reasoning_contentstring, which has no slot for a signature.This has been broken since the
truefoundryprovider type was added in #483. It is invisible on Vertex and Microsoft Foundry, which accept unsigned reasoning; only Bedrock enforces it.What the gateway actually sends
Traced delta-by-delta against a live gateway on
claude-sonnet-5:The trailing
thinking_blocksarray is already exactly ourThinkingBlockshape, so this takes it wholesale rather than pairing a loose signature with an open block. That needs no ordering logic and keeps interleaved thinking intact.Changes
truefoundryReasoning.ts: aMetadataExtractorthat pulls the gateway's consolidatedthinking_blocksoff raw chunks, plus the readers for getting them back out.compatibleModelpasses that extractor, fortruefoundryonly.mapStreamToChunksreads the signed blocks off thefinish-steppart, where extracted metadata lands, and lets them supersede the unsigned blocks built from reasoning deltas.toAssistantModelMessagereplays them via the adapter's message-levelopenaiCompatiblepassthrough. Reasoning parts are skipped in that case, so the gateway is not handed the same reasoning twice, once unsigned.ThinkingBlockSchemainstead of hand-rolledinguards. Malformed entries are still skipped rather than failing the message.No patch to
@ai-sdk/openai-compatible. Both directions use supported hooks, andpatches/stays empty.How was this tested?
truefoundryReasoning.test.ts(8 tests) drives the extractor with chunk shapes captured from the live gateway: the trailing consolidated block, interleaved thinking, streams that never sign, malformed input, and the non-streamed body.VercelAILLM.replay.test.ts(3 tests) asserts the exact outbound wire message, that the unsigned path still producesreasoning_content, and that a block signed by a different configured provider is not replayed.pnpm test(606 passing),pnpm typecheck,pnpm lint:ci,pnpm format:checkpass locally.Not verified end to end. The dev gateway has no Bedrock route (
claude-sonnet-5goes to Vertex,claude-opus-4-8to Microsoft Foundry), so confirming the loop is gone needs a Bedrock-backed Claude account. The unit tests pin the wire shape the gateway docs ask for.Checklist
pnpm test,pnpm typecheck,pnpm lint:ci, andpnpm format:checkpass locally.env.exampleupdated if configuration or behavior changed (no config change)Note
Medium Risk
Touches core LLM streaming, message replay, and multi-provider routing; mistakes could break tool loops or mishandle signatures across provider switches, though behavior is gated and heavily tested.
Overview
Fixes multi-turn agent loops on Bedrock-backed Claude through TrueFoundry and other OpenAI-compatible providers (
truefoundry,custom,fireworks,zai,together) by preserving and replaying signedthinking_blocksinstead of unsignedreasoning_contentalone.A new
openAICompatibleReasoninglayer plugs aMetadataExtractorinto@ai-sdk/openai-compatibleclients so consolidated gatewaythinking_blocksfrom trailing stream/response chunks are captured.mapStreamToChunksprefers those signed blocks over deltas accumulated during streaming, andtoAssistantModelMessageputs them back on the wire via message-levelopenaiCompatible.thinking_blockswhen signatures or redacted blocks require whole-block replay—avoiding duplicate unsigned reasoning on the same turn.ThinkingBlockUnionSchemais now a discriminated union (thinking vs redacted), and replay stays provider-bound when the session switches gateway type or configured provider name. New unit and cross-layer replay tests cover gateway wire shapes end to end.Reviewed by Cursor Bugbot for commit 701e02e. Bugbot is set up for automated code reviews on this repo. Configure here.