Skip to content

fix(browser): restore prompt submission when the Chrome window is hidden (#298)#302

Open
LeoLin990405 wants to merge 1 commit into
steipete:mainfrom
LeoLin990405:fix/hidden-window-send
Open

fix(browser): restore prompt submission when the Chrome window is hidden (#298)#302
LeoLin990405 wants to merge 1 commit into
steipete:mainfrom
LeoLin990405:fix/hidden-window-send

Conversation

@LeoLin990405

Copy link
Copy Markdown
Contributor

Fixes #298. The reporter's hypothesis was right, and this pins it to a specific commit with a live A/B.

Root cause (bisected to #282)

clickTrustedPoint — CDP Input.dispatchMouseEvent at viewport coordinates — was introduced by bcb835b (#282, "preserve Deep Research selection through submit"). git tag --contains bcb835bv0.15.1, which is exactly the reporter's regression window (0.15.0 good, 0.15.2 bad).

That commit also demoted the old synthetic DOM click to a comment-documented "last-resort fallback for unusual DOMs where the button is visible but has no useful rect". Since a real send button always has a rect, every send in 0.15.1+ now takes the trusted-click path.

Chrome delivers trusted input events only to a window that is being composited. --browser-hide-window hides the whole app via AppleScript (set visible ... to false), so the click is swallowed. Nothing errors: attemptSendButton returns true, promptSubmitted is set, and the run dies later in commit verification with Prompt did not appear in conversation before timeout — with tabUrl still at bare https://chatgpt.com/, exactly as reported.

This is the same failure mode I hit on the remote path in #289 (occluded window → send dropped); the local path never got the corresponding guard.

Live A/B (macOS, --copy-profile, gpt-5.5-pro[browser], no attachments)

Single variable: --browser-hide-window.

Current main, hidden window — reproduced:

Clicked send button
Prompt commit check failed; latest state: {"hasNewTurn":false,"composerCleared":false,"turnsCount":0,
 "inConversation":false,"href":"https://chatgpt.com/",
 "editorValue":"Reply with exactly: HIDDEN_WINDOW_SEND_OK", ...}
ERROR: Prompt did not appear in conversation before timeout (send may have failed)

The prompt is still sitting in the composer — the click never landed. (Same command with a visible window: answer captured in 21.0s.)

This branch, hidden window — fixed:

Launched Chrome (pid …) on port 9222
[browser] Focus emulation enabled for local target
Clicked send button
[browser] conversation url (post-submit) = https://chatgpt.com/c/<id>
Answer:
HIDDEN_WINDOW_SEND_FIXED
25.1s · gpt-5.5-pro[browser] · ↑16 ↓6 ↻0 Δ22

Changes

  1. Enable focus emulation on the local path. Emulation.setFocusEmulationEnabled makes the page behave like a foreground tab. The remote path already does this (fix(browser): recover remote Chrome sessions reliably #289); this factors it into a shared enableFocusEmulation(client, logger, label) helper used by both, and soft-fails with a log line if the target rejects it.

  2. Stop trusting the trusted click blindly. After dispatching it, sendTookEffect confirms the send actually left the composer (composer cleared / send button gone / stop button visible / assistant turn present, polled up to 1.5s). If it clearly did not, fall back to the synthetic DOM click that worked before fix(browser): preserve Deep Research selection through submit #282. The fallback re-queries for a visible and enabled send button — after a successful send there isn't one, so a slow-but-successful send cannot be submitted twice.

(1) alone fixes the reported case; (2) is defense in depth for any environment where trusted input cannot land (and restores 0.15.0's behavior as the floor rather than failing the run).

Answering the reporter's questions

Checks

  • pnpm run check clean; pnpm vitest run — 1421 passed, 43 skipped, 0 failed.
  • 5 new unit tests: sendTookEffect positive/negative, dispatchSendButtonClick present/absent button, and both attemptSendButton branches (fallback fires when swallowed; fallback does not fire when the trusted click landed).
  • Live runs above executed against this branch (no "Answer now" clicks; conversations archived per default).

Since steipete#282 the send button is activated with trusted CDP input events at
viewport coordinates (Input.dispatchMouseEvent), with the synthetic DOM click
demoted to a last-resort path for buttons without a usable rect. Chrome
delivers trusted input only to a composited window, so a hidden
(--browser-hide-window), minimized, or occluded window swallows the click.
attemptSendButton still reports success, promptSubmitted is set, and the run
then fails in commit verification with 'Prompt did not appear in conversation
before timeout'. 0.15.0 was unaffected because it always used the DOM click.

Two changes:

1. Enable Emulation.setFocusEmulationEnabled on the local browser path, so the
   page behaves like a foreground tab. This already guards the remote path
   (steipete#289); factor it into a shared enableFocusEmulation helper used by both.

2. Stop trusting the trusted click blindly. After dispatching it, confirm the
   send actually left the composer (composer cleared / send button gone / stop
   button / assistant turn); if it clearly did not, fall back to the synthetic
   DOM click that worked before steipete#282. The fallback re-queries for a visible and
   *enabled* send button, so a slow-but-successful send cannot be sent twice.

Live A/B on macOS with --browser-hide-window (copy-profile, gpt-5.5-pro):
- before: 'Clicked send button' then commit-timeout; commitProbe shows the full
  prompt still in the composer, turnsCount 0, url never leaves chatgpt.com/
- after:  'Focus emulation enabled for local target' -> send commits -> /c/<id>
  conversation -> answer captured in 25.1s

Fixes steipete#298
@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P1 Urgent regression or broken agent/channel workflow affecting real users now. merge-risk: 🚨 message-delivery 🚨 Merging this PR could drop, duplicate, misroute, suppress, or wrongly target messages. labels Jul 10, 2026
@clawsweeper

clawsweeper Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs changes before merge. Reviewed July 10, 2026, 5:43 AM ET / 09:43 UTC.

Summary
Enables local browser focus emulation and adds a synthetic DOM-click fallback when a trusted CDP send click appears ineffective.

Reproducibility: yes. The PR supplies a controlled current-main failure and branch success for hidden-window submission, and source inspection establishes a deterministic false-positive path when a conversation already contains assistant messages.

Review metrics: 2 noteworthy metrics.

  • Changed surface: 226 added, 7 removed across 3 files. The patch modifies shared prompt delivery, browser-target initialization, and focused unit coverage.
  • Validation reported: 1 live A/B and 5 new unit tests. The root-page hidden-window behavior is well demonstrated, but no test covers a continued conversation with prior assistant messages.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #298
Summary: This PR is the direct candidate fix for the canonical hidden-window send regression and builds on related trusted-click and remote-focus work.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🦞 diamond lobster
Patch quality: 🦐 gold shrimp
Result: needs maintainer review before merge.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Make send-effect detection baseline-aware and add a swallowed-click regression test with pre-existing assistant messages.
  • Repeat the hidden-window proof in a continued conversation if focus emulation remains intentionally soft-failable.

Risk before merge

  • [P1] When focus emulation soft-fails or is ineffective, a continued conversation containing an older assistant message can suppress recovery of a swallowed click and leave the new prompt undelivered.
  • [P2] Because the fallback operates directly in prompt delivery, success signals must represent a new post-click transition rather than historical DOM content.

Maintainer options:

  1. Baseline the effect probe (recommended)
    Capture the pre-click assistant or turn state and require a newly observed transition before suppressing the synthetic fallback.
  2. Land focus emulation only
    Remove the defense-in-depth fallback and retain only the proven local focus-emulation change if transition-safe detection is not ready.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Fix the send-effect probe so pre-existing assistant turns in an existing conversation do not count as evidence that the new trusted click landed; add focused regression coverage for a swallowed click with prior assistant messages, while preserving the hidden-window live fix.

Next step before merge

  • [P2] The blocker has a narrow code-and-test repair with no unresolved product, ownership, or security decision.

Security
Cleared: The patch introduces no dependencies, downloaded code, permission changes, secret handling, or publishing and supply-chain changes.

Review findings

  • [P1] Baseline assistant turns before treating them as send success — src/browser/actions/promptComposer.ts:814-815
Review details

Best possible solution:

Retain the proven local focus-emulation fix, but compare post-click state with a pre-click baseline so the fallback fires exactly once when no new submission transition occurs.

Do we have a high-confidence way to reproduce the issue?

Yes. The PR supplies a controlled current-main failure and branch success for hidden-window submission, and source inspection establishes a deterministic false-positive path when a conversation already contains assistant messages.

Is this the best way to solve the issue?

No, not yet. Focus emulation is the narrow proven fix, but the fallback must detect a new attributable state transition rather than treating historical assistant content as success.

Full review comments:

  • [P1] Baseline assistant turns before treating them as send success — src/browser/actions/promptComposer.ts:814-815
    assistantVisible is already true whenever a continued conversation contains an older assistant reply, so sendTookEffect() immediately reports success even if the trusted click was swallowed and the new prompt remains in the composer. This prevents the fallback when focus emulation soft-fails or is ineffective; compare against a pre-click turn baseline or require a newly observed transition.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.97

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f3fbc067c9d8.

Label changes

Label changes:

  • add P1: The PR addresses a released browser-mode regression that prevents prompt submission, but its recovery path remains broken for continued conversations under fallback conditions.
  • add merge-risk: 🚨 message-delivery: Historical assistant DOM can be mistaken for a successful new submission, suppressing recovery and leaving the prompt undelivered.
  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body contains a controlled current-main failure and after-fix hidden-window live run showing conversation navigation and captured output, while explicitly confirming Answer now was not used.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
  • add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (live_output): The PR body contains a controlled current-main failure and after-fix hidden-window live run showing conversation navigation and captured output, while explicitly confirming Answer now was not used.

Label justifications:

  • P1: The PR addresses a released browser-mode regression that prevents prompt submission, but its recovery path remains broken for continued conversations under fallback conditions.
  • merge-risk: 🚨 message-delivery: Historical assistant DOM can be mistaken for a successful new submission, suppressing recovery and leaving the prompt undelivered.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (live_output): The PR body contains a controlled current-main failure and after-fix hidden-window live run showing conversation navigation and captured output, while explicitly confirming Answer now was not used.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body contains a controlled current-main failure and after-fix hidden-window live run showing conversation navigation and captured output, while explicitly confirming Answer now was not used.
Evidence reviewed

Acceptance criteria:

  • [P1] pnpm vitest run tests/browser/promptComposer.test.ts.
  • [P1] pnpm run check.

What I checked:

  • Blocking existing-conversation path: The effect probe treats any assistant-role element as evidence that the current send succeeded, without comparing against a pre-click baseline; prior assistant messages therefore suppress the fallback even when the new prompt remains in the composer. (src/browser/actions/promptComposer.ts:814, 8e4dd1ab0a73)
  • Regression provenance: History ties the trusted CDP send-click path to the merged Deep Research submission change, matching the reported release regression window. (src/browser/actions/promptComposer.ts:722, bcb835b27010)
  • Related focus-emulation precedent: The remote-browser recovery change previously introduced focus emulation for the adjacent background-target failure mode reused by this PR. (src/browser/index.ts:2840, d3a7df0f137b)
  • Real behavior proof: The PR body provides a controlled hidden-window A/B: current main leaves the prompt in the composer, while the branch navigates to a conversation and captures the requested answer without using Answer now. (8e4dd1ab0a73)
  • Current checks: GitGuardian and the macOS, Ubuntu, and Windows build jobs pass on the reviewed head, although they do not exercise the prior-assistant-message fallback case. (8e4dd1ab0a73)
  • Repository browser-proof policy: The full AGENTS.md was read; the submitted proof follows its browser guidance by capturing the actual assistant output and avoiding the Answer now shortcut. (AGENTS.md:1, f3fbc067c9d8)

Likely related people:

  • LeoLin990405: Prior merged work added prompt-commit diagnostics and remote focus emulation in the same browser paths, establishing relevant current-main history beyond this proposal. (role: recent area contributor; confidence: high; commits: 08c5587f58a7, d3a7df0f137b; files: src/browser/actions/promptComposer.ts, src/browser/index.ts)
  • wbzjt: The trusted CDP send-click behavior adjusted here originated in the merged Deep Research submission change. (role: introduced trusted-click behavior; confidence: high; commits: bcb835b27010; files: src/browser/actions/promptComposer.ts, src/browser/actions/deepResearch.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 message-delivery 🚨 Merging this PR could drop, duplicate, misroute, suppress, or wrongly target messages. P1 Urgent regression or broken agent/channel workflow affecting real users now. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Browser send regression in 0.15.2: prompt never submitted with hidden window (manual-login); works in 0.15.0

1 participant