Fix #279: worker-queue ESC-cancel for Anthropic/Minimax providers#311
Open
ericleepi314 wants to merge 1 commit into
Open
Fix #279: worker-queue ESC-cancel for Anthropic/Minimax providers#311ericleepi314 wants to merge 1 commit into
ericleepi314 wants to merge 1 commit into
Conversation
…max (#279) The close-on-abort listener alone is best-effort: behind a buffering proxy (LiteLLM, corporate proxy, mitmproxy) the anthropic SDK's sync httpx read doesn't unblock on a cross-thread response.close(), so ESC hung — the same failure the OpenAI-compatible path fixed in PR #148. - extract the #278-hardened worker+bounded-queue pattern into src/providers/_stream_worker.py (run_stream_on_worker): emit/on_item relay, bounded queue, drop-on-abort/consumer-gone, error relay that preserves exception chaining and KeyboardInterrupt semantics - openai_compatible: replaced inline machinery with the helper (behavior pinned by the existing 11 abort tests) - anthropic: only the iteration + get_final_message move to the worker; stream lifecycle, guard.attach, watchdog arm/fallback stay on the consumer thread, with a consumer-side disarm guarantee so a stuck worker can't leak the 90s timer; watchdog state crosses threads via a dict written before the queue relay - minimax: same shape minus the watchdog Closes #279 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #279
Summary
PR #148's worker-thread + queue ESC-unwind only covered
OpenAICompatibleProvider; Anthropic and Minimax relied solely on the close-on-abort listener, which is best-effort — behind a buffering proxy (LiteLLM, corporate, mitmproxy) the anthropic SDK's sync httpx read doesn't unblock on a cross-threadresponse.close(), so ESC hung.src/providers/_stream_worker.py—run_stream_on_worker(produce, on_item, guard): the OpenAICompatibleProvider ESC-cancel uses an unbounded queue.Queue; pathological streams can accumulate megabytes after ESC #278-hardened pattern (bounded queue at 64, drop-on-abort,consumer_gonerelease, items-before-abort delivered / nothing after) generalized so each provider only owns its SDK iteration shape. Error relay preserves exception chaining (AbortError from original) and re-raisesKeyboardInterrupt/SystemExitas-is.get_final_messagemove to the worker. Stream lifecycle,guard.attach, and the WI-5.2 watchdog arm/fallback stay on the consumer thread; a consumer-sidefinally: watchdog.disarm()guarantees the 90s timer can't leak when ESC abandons a stuck worker (critic find). Watchdog state crosses threads via a dict written before the queue relay (queue mutex is the happens-before edge).Test plan
_Stuck*Streamregression tests for both providers (iterator never honorsclose(), blocks on a never-set Event): ESC unwinds in <3s where it previously hung forever🤖 Generated with Claude Code