Skip to content

fix(agent-runtime): stop waiting for tools host-core cannot cancel - #1779

Open
liubai0106 wants to merge 2 commits into
vastsa:mainfrom
liubai0106:fix/tool-abort-grace
Open

liubai0106 wants to merge 2 commits into
vastsa:mainfrom
liubai0106:fix/tool-abort-grace

Conversation

@liubai0106

@liubai0106 liubai0106 commented Oct 20, 2026 •

Copy link
Copy Markdown

What

Stop now ends the turn even when the running tool cannot be cancelled, and the stop RPC always returns.

Two independent holes in packages/agent-runtime/src/runtime.ts:

1. The tool executor ignored the stop for every tool but Bash / GenerateImages. The abort listener returns immediately unless the tool is one of those two (runtime.ts:3470 on main), and tools.execute is awaited with no deadline and no race (runtime.ts:3517). One report measured a Grep running 21m39s after six Stop clicks, with the abort applying in the same millisecond it finished.

2. abort() waits for the turn settlement with no bound. It ends with await settlement (runtime.ts:9100 on main), and the settlement only resolves once the turn unwinds. A tool that never yields therefore keeps the agent.abort RPC pending forever — and the renderer awaits that RPC before it clears the running state (apps/desktop/src/stores/slices/transcript-slice.ts:501, await Promise.allSettled([api.abort(sessionId), ...])), so the session stays at "processing" even after the ledger has recorded the turn as aborted. The ledger and the UI disagree, and only a restart reconciles them.

Changes

  1. The listener applies to every tool. The (!isBash && toolName !== "GenerateImages") guard is gone.
  2. The wait races the signal. tools.execute is raced against an abort promise.
    • Bash / GenerateImages: still send tools.abort, then wait a bounded 10s grace window for the partial output the host collected before the kill.
    • Every other tool: abandon the wait the instant the signal fires and return TOOL_ABORTED.
  3. Losing the race abandons the wait, not the host call. The orphaned tools.execute settles on its own; its result is discarded and never surfaces.
  4. abort() drives the in-flight tools directly. It walks the active abort handlers and runs the same path a signal abort would, instead of relying only on the signal each tool was handed. A stop must not depend on a signal the runtime does not own: if the signal never fires, the stop is a silent no-op and abort() blocks on the settlement, which is the failure in point 2 above.

Why non-cancellable tools do not send tools.abort

tools.abort only clears a pending approval and kills a Bash / GenerateImages process — for anything else host-core answers found: false and the tool keeps running. On top of that, an abort for a call id with no active cancellation registers a 60s tombstone in pending_bash_aborts (crates/host-core/src/state.rs), which a later Bash reusing that id would inherit as an immediate kill. So the RPC cannot help the in-flight call and can only hurt a retry, which matches the note in the issue: send tools.abort for Bash/image tools as today, rely on the local race for the rest, and require a distinct id on retry.

Docs

  • docs/adr/tool-abort-abandons-host-call.md — new ADR: the abandoned call keeps consuming host resources, the result is discarded, a retry must carry a fresh tool call id, and abort() releases tools itself rather than trusting the signal.
  • docs/spec/03-runtime/03-tools-and-permissions.md §3 — the stop contract, including the 10s grace window for cancellable tools and the guarantee that the stop RPC returns.

Tests

packages/agent-runtime/src/runtime.test.ts:

  • ends the turn when a stop lands while a tool host-core cannot cancel runs — a Grep whose host call never resolves rejects with TOOL_ABORTED, no tools.abort is sent, and the orphaned call settling later produces no second result.
  • stops waiting for a Bash call that ignores tools.abort once the grace window closes — a hung Bash sends tools.abort and rejects with TOOL_ABORTED when the 10s window closes.
  • releases an in-flight tool when the stop lands without the tool signal firing — abort() ends a pending Grep whose signal is never aborted, and the signal is still un-aborted afterwards, so the release came from abort() and not from the signal.

332 tests pass in runtime.test.ts, as do pnpm typecheck, pnpm lint, pnpm check:pr-base, and pnpm check:agent-policy.

No E2E scenario: it needs a tool that reliably runs longer than the test budget, and no built-in tool can be made to hang on demand without a host-core fixture.

Residual

A non-cancellable tool keeps running in host-core after the stop. That is real work, not a stuck UI, and the durable fix stays on the Rust side — teach tools.execute an abort token so Grep and friends can actually be cancelled. A Grep that can run 21 minutes is worth filing on its own, as the issue notes.

Process-tree reclamation is a separate host-core gap: tools.abort terminates the job it owns, but a shell launcher that re-executes the real interpreter can leave the grandchild running. Out of scope here.

Fixes #1767

@liubai0106
liubai0106 force-pushed the fix/tool-abort-grace branch from a390dbb to 9f04861 Compare October 20, 2026 08:20
liubai added 2 commits October 20, 2026 19:53
The abort listener returned immediately unless the tool was Bash or
GenerateImages, so Stop was a no-op for every other tool, and
tools.execute was awaited with no deadline. One report measured a Grep
running 21m39s after six Stop clicks.

The listener now applies to every tool. Bash and GenerateImages keep
sending tools.abort and get a bounded 10s grace window for the partial
output the host collected before the kill; every other tool abandons the
wait the moment the signal fires and returns TOOL_ABORTED without a
tools.abort RPC (the host cannot act on it, and the 60s tombstone it
would leave could be inherited by a later Bash reusing the call id).

Both paths race the execute call against the abort, so losing the race
abandons the wait rather than the host call: the orphaned call settles on
its own and its result is discarded.

Fixes vastsa#1767
A stop can reach the runtime without reaching the tool it was meant to
cancel. The executor then keeps awaiting `tools.execute`, the turn never
settles, and `abort()` — which awaits that settlement — never returns. The
renderer awaits the abort RPC before it clears "processing", so the session
stayed pinned even though the ledger had already recorded the turn as
aborted: three turns in one field report were `aborted` while the tool's
process was still alive minutes later.

`abort()` now drives the in-flight tools directly instead of relying only on
the signal each one was handed: it walks the active abort handlers and runs
the same path a signal abort would. Cancellable tools still send
`tools.abort` and get the grace window; every other tool abandons the wait
at once.

Refs vastsa#1767
@liubai0106
liubai0106 force-pushed the fix/tool-abort-grace branch from 9f04861 to 984367f Compare October 20, 2026 11:55

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Stopping a turn does not interrupt long-running tools (Grep ran 21m39s after six stop clicks)

1 participant