Skip to content

fix(python): forward binary tool results in HandlePendingToolCall RPC#1821

Open
syf2211 wants to merge 1 commit into
github:mainfrom
syf2211:fix/1644-forward-binary-tool-results
Open

fix(python): forward binary tool results in HandlePendingToolCall RPC#1821
syf2211 wants to merge 1 commit into
github:mainfrom
syf2211:fix/1644-forward-binary-tool-results

Conversation

@syf2211

@syf2211 syf2211 commented Jun 28, 2026

Copy link
Copy Markdown

Summary

When a Python tool handler returns a ToolResult with binary_results_for_llm (e.g. base64 image data) or session_log, those fields were dropped while building the ExternalToolTextResultForLlm payload in _execute_tool_and_respond. The model only received text fields, so image tool outputs appeared unavailable.

Motivation

Fixes #1644. Users building agents with image-returning tools saw the model report it could not view images, even though direct blob attachments worked.

Changes

  • Add tool_result_to_external_tool_text_result_for_llm() in python/copilot/tools.py to map ToolResultExternalToolTextResultForLlm, including binary result type enum conversion.
  • Use the helper in CopilotSession._execute_tool_and_respond instead of an inline constructor that omitted binary_results_for_llm and session_log.
  • Add unit tests covering binary forwarding and the no-binary case.

Tests

cd python && uv run pytest test_tools.py::TestToolResultToExternalToolTextResultForLlm -q
cd python && uv run ruff check copilot/tools.py copilot/session.py test_tools.py
  • 2 passed
  • ruff: All checks passed

Notes

  • Scope is Python SDK only; Node passes the full result object through RPC already.
  • Composer-2.5 code review: APPROVE

ToolResult.binary_results_for_llm and session_log were dropped when
building ExternalToolTextResultForLlm in _execute_tool_and_respond,
so image/binary tool outputs never reached the model.

Fixes github#1644
@syf2211 syf2211 requested a review from a team as a code owner June 28, 2026 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Binary tool results not passed to ExternalToolTextResultForLlm

1 participant