Skip to content

fix: report failed actions as MCP tool errors - #280

Merged
avifenesh merged 3 commits into
agent-sh:mainfrom
OnlyHa1bara:ha1bara/fix-mcp-action-error-status
Oct 24, 2026
Merged

avifenesh merged 3 commits into
agent-sh:mainfrom
OnlyHa1bara:ha1bara/fix-mcp-action-error-status

Conversation

@OnlyHa1bara

@OnlyHa1bara OnlyHa1bara commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Why

Failed desktop actions returned ok: false in their structured output, but the shared result builder reported isError: false. MCP clients could therefore treat refusals and execution failures as successful tool calls.

What Changed

Set the MCP error flag when the shared result builder receives an explicit top-level ok: false. Extend existing assertions for failed actions and observation-only results. Successful actions remain successful when only post-action capture fails.

Update the existing GTK click regression and Electron input reproduction to expect isError: true for intentional refusals, while preserving the success, payload, refusal-reason, and UI outcome checks.

Validation

  • GTK/MCP regression in an isolated Xvfb display: reproduced the assertion failure at the expected pointer refusal before updating the script; the updated script passes at scales 1 and 2, including both refusal cases and actual button activation.
  • Electron script: syntax check and eight focused response cases pass, covering normal success, expected password refusal, incorrect error flags, unexpected success/failure, and an unrelated refusal reason.
  • Full native Hyprland/Electron reproduction was not run because Hyprland is unavailable in this environment.

@OnlyHa1bara OnlyHa1bara changed the title [codex] Report failed actions as MCP tool errors fix: report failed actions as MCP tool errors Oct 22, 2026
@OnlyHa1bara
OnlyHa1bara marked this pull request as ready for review October 22, 2026 16:18

@revuto-review revuto-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an auto review done by revuto.


Tying isError to ok in structured_tool_result fixes the protocol problem described in the PR. It also changes what clients see for every refusal that comes back as ok:false. Two of the repo's own MCP regression scripts (scripts/semantic_click_test.py and scripts/repro_wtype_electron.cjs) still assume refusals return isError:false. They will now fail on their expected-refusal cases. Update them in this PR.

Comment thread src/server.rs
})?;
let is_error = value.get("ok").and_then(serde_json::Value::as_bool) == Some(false);
let mut result = CallToolResult::structured(value);
result.is_error = Some(is_error);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an auto review done by revuto.


With this change, every ok:false result now comes back as isError: true. That includes refusals that are part of the safety contract, like password-field rejection and a disabled pointer fallback, not just real execution failures. Two of the repo's own MCP regression scripts still expect the old isError: false and will now fail:

  • scripts/semantic_click_test.py: the shared rpc() helper runs assert not result.get('isError'), result (line 34) on every response. Lines 118–123 then make clicks that are supposed to be refused (click {x:0,y:0} and the entry element_index click with pointer backends disabled) and check not ...['ok']. Those calls now return isError: true, so the assert in rpc() fails before the ok check runs.
  • scripts/repro_wtype_electron.cjs:67: the mcp-password mode passes only when !response.isError && payload.ok === false && payload.message.includes('password fields'). type_text sends its Err(...) from dispatch_type_text through finish_action_output and then this builder. The password refusal now sets isError: true, so that mode exits 1.

The Rust unit tests and the CI list in the PR description don't run these scripts, which is why this wasn't caught. Please update both scripts in this PR: allow isError when the expected result is ok:false, and require isError === true there. A CHANGELOG entry would also help, since hosts that handle isError differently from ok will see a change in behavior.

@OnlyHa1bara
OnlyHa1bara force-pushed the ha1bara/fix-mcp-action-error-status branch from 4a2f750 to bfeb1a0 Compare October 22, 2026 16:41

@revuto-review revuto-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an auto review done by revuto.


Revuto completed the review and found no evidence-backed concerns.

@avifenesh

Copy link
Copy Markdown
Collaborator

Thanks for the fix, @OnlyHa1bara — LGTM!

@avifenesh
avifenesh merged commit a99f770 into agent-sh:main Oct 24, 2026
17 of 18 checks passed
@avifenesh

Copy link
Copy Markdown
Collaborator

Released in 0.7.23. Thanks @OnlyHa1bara for the MCP error-result fix. The final release passes the MCP safety/schema checks and keeps successful actions successful when only post-action observation fails.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants