fix: streaming parallel tool-use emit logic (issue #197)#200
Conversation
RawContentBlockStopEvent was iterating over the entire streamingFunctions dictionary and clearing it. When a later tool_use block's content_block_start arrived before an earlier block's content_block_stop, the stop event would emit FunctionCallContent for every tracked tool — including ones whose input_json deltas had not yet arrived — and then wipe the dictionary, losing the later tools' real arguments. Only emit (and remove) the entry whose index matches the stop event, in both AnthropicClientExtensions and AnthropicBetaClientExtensions. Add a regression test in AnthropicClientExtensionsTestsBase so coverage applies to both client variants. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| data: {"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\"arg\":\"a\"}"}} | ||
|
|
||
| event: content_block_start | ||
| data: {"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"toolu_b","name":"tool_b","input":{},"caller":{"type":"direct"}}} |
There was a problem hiding this comment.
@ziv690922 what's still odd to me is how you could get tool_use ids must be unique if the ids are part of the content_block_start, they should never be duplicated, even in the buggy code.
There was a problem hiding this comment.
You're right, the duplicate ids are not caused by this bug. After further investigation, the tool_use ids must be unique error was coming from FunctionInvokingChatClient internally appending the assistant turn (including tool_use blocks) into the messages list across iterations, which resulted in duplicate ids being sent to the Anthropic API. That's a separate issue from what this PR fixes.
|
@stephentoub does this look right to you? |
TomerAberbach
changed the title
|
@TomerAberbach, sorry for the delay. This looks good to me. Also, for future reference, both @jozkee and @jeffhandley are on my team at Microsoft and I trust them with these kinds of fixes and improvements. |
5 participants
RawContentBlockStopEvent was iterating over the entire streamingFunctions dictionary and clearing it. When a later tool_use block's content_block_start arrived before an earlier block's content_block_stop, the stop event would emit FunctionCallContent for every tracked tool, including ones whose input_json deltas had not yet arrived, and then wipe the dictionary, losing the later tools' real arguments.
Only emit (and remove) the entry whose index matches the stop event, in both AnthropicClientExtensions and AnthropicBetaClientExtensions. Add a regression test in AnthropicClientExtensionsTestsBase so coverage applies to both client variants.
Fixes #197