fix(python): forward binary tool results in HandlePendingToolCall RPC#1821
Open
syf2211 wants to merge 1 commit into
Open
fix(python): forward binary tool results in HandlePendingToolCall RPC#1821syf2211 wants to merge 1 commit into
syf2211 wants to merge 1 commit into
Conversation
ToolResult.binary_results_for_llm and session_log were dropped when building ExternalToolTextResultForLlm in _execute_tool_and_respond, so image/binary tool outputs never reached the model. Fixes github#1644
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a Python tool handler returns a
ToolResultwithbinary_results_for_llm(e.g. base64 image data) orsession_log, those fields were dropped while building theExternalToolTextResultForLlmpayload in_execute_tool_and_respond. The model only received text fields, so image tool outputs appeared unavailable.Motivation
Fixes #1644. Users building agents with image-returning tools saw the model report it could not view images, even though direct blob attachments worked.
Changes
tool_result_to_external_tool_text_result_for_llm()inpython/copilot/tools.pyto mapToolResult→ExternalToolTextResultForLlm, including binary result type enum conversion.CopilotSession._execute_tool_and_respondinstead of an inline constructor that omittedbinary_results_for_llmandsession_log.Tests
Notes