Clean test VM: click Open on the download prompt over VNC, like a user

[claude]

Before: in run 4 on Justin's Mac, the Setup Assistant fix (#1822) worked, the desktop came up clear, and the single VNC connection held. But Transcripted never started. The "downloaded from the Internet… Are you sure you want to open it?" prompt stayed on screen for the whole run. approve-download cleared the quarantine flag, but that doesn't close a prompt that's already showing, and the second launch only brought the prompt forward. So app_launched timed out.

After: approve-download clicks the prompt's Open button over the VNC session (the virtual mouse), which is what a real user does. Every click lands inside the prompt's own window, read from the guest each time. Return is pressed only while that prompt is the front window. Before each retry it checks whether the app has already started. Clearing the quarantine flag is only a last resort. It exits 3, so first-run reports that step as "ok via bypass" and the summary never says "all steps ok" after it.

How:

• vnc.py click-default-button [--dry-run] [--within X,Y,W,H --points-wide N] takes a screenshot over the existing session and finds the blue default button. It looks for a solid, pill-shaped patch of macOS's accent blue above the Dock, with limits on size, aspect ratio and fill. With --within, it only takes a button whose centre is inside that window, with the box scaled from screen points to framebuffer pixels.
• approve-download:
  • It runs the Gatekeeper spctl --assess check first, as before.
  • It reads the guest's window list with osascript -l JavaScript and CGWindowListCopyWindowInfo. That gives window owners and bounds only, so no Accessibility or Screen Recording grant is needed. It finds the CoreServicesUIAgent prompt window and which window is in front.
  • It tries clicking Open twice inside that window, then Return if the prompt is in front. Each try first checks for the Transcripted process.
  • If it can't read the window list, it doesn't click blind.
  • With no screen access, no readable window list, or a prompt that doesn't respond, it falls back to clearing the flag, which exits 3.
• Review follow-ups:
  • vnc.py refuses keysyms above 0xFFFFFFFF.
  • serve treats a struct packing error as a failed command instead of dying (N1 from Clean test VM: one VNC connection per boot, VNC off by default #1820's post-merge check).
  • The stale comment above approve-download is gone.

Tests:

• find_default_button against the real screenshots from runs 1–4:
  • It finds Open at (487, 238) in all three run 4 frames (900×562).
  • It finds (1558, 764) on a 2880×1800 upscale of the same frame.
  • It finds nothing in the five other real frames.
• vnc.py --self-test covers:
  • a synthetic prompt, where blue Dock icons, a dot and sky are ignored
  • --within choosing the prompt's button over a bigger one elsewhere, and returning none outside it
  • point-to-pixel scaling
  • an out-of-range keysym over serve, which fails the request while the session stays up
• test-transcripted-vm.sh is at 58/58, with new fake-guest cases:
  • an app that's already running is left alone
  • with no window list, nothing is clicked or pressed, and it exits 3
  • a prompt that isn't in front gets no Return, and it exits 3
  • a slow start is caught before the next retry
  • Return on a front prompt counts as the user path
  • Mutation checks confirm the retry-check and front-window tests fail when those guards are removed.
• supervise.py --self-test, shellcheck -S warning, scripts/dev/linux-checks.sh (49/49) and check-source-pins.py --changed-only all pass.

Run 5 will show whether the window list reads the same way on macOS 26 and whether the click starts the app. The report says which path it took. No .github/workflows changes.

🤖 Generated with Claude Code

https://claude.ai/code/session_011CFGqqwpKNjdBifQbmEMsS

---

Generated by Claude Code