Skip to content

feat(dialog): let agents accept, dismiss and fill JavaScript dialogs - #378

Open
ArcueidMP wants to merge 12 commits into
Tencent:mainfrom
ArcueidMP:codex/issue-359-js-dialogs
Open

ArcueidMP wants to merge 12 commits into
Tencent:mainfrom
ArcueidMP:codex/issue-359-js-dialogs

Conversation

@ArcueidMP

@ArcueidMP ArcueidMP commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Native confirm() and prompt() dialogs are currently accepted automatically, so agents cannot cancel a confirmation or supply prompt input. This change leaves those dialogs pending and adds explicit status, accept and dismiss commands.

Fixes #359.

  • Add bsk dialog status, bsk dialog accept [text] and bsk dialog dismiss, with optional tab selection and a dialog ID to reject stale decisions. Omitted prompt text preserves the default; an empty string clears it.
  • Keep automatic handling for alert and beforeunload by default. session start --no-auto-dialog leaves all four types pending, including a beforeunload choice to stay on the page.
  • Return dialog_pending promptly when a dialog blocks a tool, including lifecycle waits. Dialog control can run while the session is busy and still enforces ownership and user interruption. Track the actual native command until it settles, preserve cleanup without replaying input, and handle manual closure, consecutive dialogs and debugger detach.
  • Advertise protocol 1.4, reject unsupported opt-out requests on older components, and add matching DSH actions, schemas, skill guidance and regressions picked up by the existing browser-input CI command.

The pending response ends the original CLI wait, not execution inside Chrome. Handling the dialog may resume the already-sent native operation; it is never replayed. The original evaluation return value is not recovered. Callers inspect the page afterwards and wait while execution_pending is true.

The video encoder regression also waits for frame presentation after seeking before sampling pixels; its existing color assertions are unchanged.

Local validation:

  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets --locked -- -D warnings
  • cargo test --workspace --locked
  • pnpm lint
  • pnpm --filter @browser-skill/extension compile
  • pnpm ext:test and pnpm ext:build
  • node --test scripts/*.test.mjs, including LF/CRLF skill budget checks
  • node scripts/check-skill-bundles.mjs and node scripts/check-crate-skill.mjs
  • DSH plugin build and node scripts/check-dsh-package.mjs against an actual npm archive
  • 8 isolated Chrome regressions across click-dialogs.browser.test.ts and click.browser.test.ts. The dialog cases are included by the existing CI glob.
  • Video encoder/recovery and capture/export browser regressions with isolated Chrome for Testing. The headed preview test remains opt-in.

Resolve the SKILL.md reference-table conflict by listing both JS dialogs
and the new element waits for the interaction-details reference. The
entry point stays within its byte budget.
Keep both JavaScript dialog control and the new video tool: register both
DSH tools and schemas, mark both as control-plane methods, and gate dialog
decisions and video start on user interrupts.

Tighten the DSH skill entry point so the combined dialog and video guidance
stays within its 4,500-byte budget after Windows CRLF checkout.
Expose both wait-for-element and dialog on the DSH browser_page tool, keeping
each action's guidance in the tool description and both in the expected
action maps.
Keep JavaScript dialogs alongside structured extraction: register both DSH
tools and list both sets of error reasons. In the CLI skill, take main's
singular ref wording but keep the shorter reference-loading sentence so the
entry point stays within its 7,100-byte budget after Windows CRLF checkout.
The large-frame encoder check failed on a final-frame colour sample. The test only bundles video modules and type-only dialog changes; rerun to separate CI timing from a branch regression.
Adopt the DSH bsk_* tool names for the dialog action: tool maps, skill
entry point, interaction reference and session-resolution label now use
bsk_page/bsk_session. List dialog in the README's bsk_page actions, and
keep both the Added dialog entry and the Changed namespace entry.
Keep --no-auto-dialog alongside the new background-by-default focus
handling: the CLI sends both focused and no_auto_dialog, and the DSH start
tool maps noFocus to --focus/--no-focus and noAutoDialog to
--no-auto-dialog. Keep the dialog parse test before main's renamed
background session test.
The focus mapping cases from Tencent#447 still looked up browser_session after Tencent#446 renamed the tool, so they failed on main as well.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Give agents control over JS dialogs (confirm/prompt/beforeunload): dismiss/text/status instead of forced auto-accept

1 participant