Skip to content

The bot's browser keeps up with your mouse: input on one socket, the newest move wins - #183

Merged
TigerYWang merged 22 commits into
mainfrom
dev/browser-speed
Nov 12, 2026
Merged

TigerYWang merged 22 commits into
mainfrom
dev/browser-speed

Conversation

@abhaypadmanabhan

Copy link
Copy Markdown
Contributor

Moving the mouse in a bot's browser, then clicking, could take seconds to show. Every move, click and key went to Bops Cloud as its own POST (a sign-in check and a database read each), and the next one waited for the answer. Moves at 20 a second queued behind each other, so a click waited behind them.

Now:

  • One socket per bot for input (WS /v1/browser/input). Who may use the bot's browser is asked at the upgrade, again every ROLE_MS (the key too), and at once on orgo-web's access notice, as the watch stream is. Each act gets the same checks as a POST, and BROWSER_METHODS still applies at Cloud and at the runner.
  • The Mac's server keeps one socket per person, workspace and bot, and answers the window as soon as an act is on its way. An older Bops Cloud (a plain 404 on the upgrade) gets every act by POST, as before. Take, hand back and refused methods still go by POST, so their refusals keep Bops Cloud's words.
  • The window sends moves at about 60 a second and replaces a waiting move or scroll with the newest one. Presses, releases and keys are never dropped or reordered. Your own press rings at once where you pressed.

Shared workspaces and several people. Nothing changes about who may drive: each person's socket is checked as them, and one person drives at a time (the owner's take goes over a member's). Two fixes found while measuring several viewers:

  • Second window: someone already watching who opened the browser again (a second Mac, or a second window) saw a blank panel until the page changed. Bops Cloud now says joined when a stream opens, and the runner sends the latest frame. Older runners ignore it.
  • Revoked key: an owner watching their own bot was never asked about their key again, so a revoked key kept seeing the bot's signed-in pages. The watch stream now asks again at each look, through the same check as the input socket.

Proof (local two-person stack, a proxy adding 35 ms each way, same probe before and after; median / p90)

main this change
Click, from rest 205 / 214 ms 121 / 135 ms
Click, right after moving 1784 / 1866 ms 117 / 133 ms
Click, after 6 s idle 283 / 293 ms 134 / 136 ms
Inputs waiting at the click after moving 18 1
Scroll, frames a second (the probe's wheel driver) 13 26 (57 with wheels at 60 a second)
A Member driving in the shared workspace, click after moving 1716 ms 116 ms
The other person watching sees it 1717 ms 120 ms
Three viewers (Member drives, owner's two windows watch), click after moving n/a (second window blank) 117 ms; each watcher 121 ms

On the real path (installed Bops, bops.orgo.ai, an Orgo computer, screen-recorded), main took 453 ms per click from rest, and about 3 s right after moving. The real path with this change needs Bops Cloud and an app release to measure. Orgo's computer accounts for about 70 ms of that path (21 ms from input to JPEG on the computer itself).

  • node scripts/verify browser-input: the input socket against a fake Cloud, Bops Cloud browser tests 16/16, both TypeScript checks, eslint on the touched files.
  • node scripts/test-browser-relay.mjs 15/15; node scripts/test-browser-tab-e2e.mjs 17/17 (the owner and a Member, take over, sign-in, hand back, tabs, links).
  • Each new test fails without its change: the socket's checks, the key re-checks (socket and stream), the second window's catch-up, and the own-press ring.
  • scripts/probe-browser-speed.mjs (--driver sam --watchers 2 for several people) and scripts/verify browser-speed: the measurement, kept so it can be rerun. Timing logs are off unless BOPS_BROWSER_PROBE=1, and never log typed text.

Order of release: any. An app on the old Cloud uses POST, and the old app on the new Cloud does too. The speedup arrives once both are live.

Reviewed with sol-loop: 3 rounds with gpt-6.1-sol. Sol verdict: approved

🤖 Generated with Claude Code

abhaypadmanabhan and others added 17 commits November 7, 2026 23:37
WS /v1/browser/input?bot=<id>[&project=<id>] carries one viewer's acts in
order. Who may is asked as the watch stream asks it: mayUseBrowser at the
upgrade (refused with its status, marked x-bops-browser-input so a plain 404
means an older Bops Cloud), again every ROLE_MS, and ended at once by
orgo-web's notice. Each act is checked as POST /v1/browser/act checks it
(shared actOf/sendAct), at most 64 KiB, and sent to the runner of that
computer at that moment; what didn't go is said back on the socket.
POST /v1/browser/act is unchanged for older Bops apps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
actBrowser sends each input on one WS /v1/browser/input per (person and
key, Bops Cloud, workspace, bot) and answers 202 once it's on its way,
instead of after a POST round trip and a database read at Bops Cloud.
Acts that come while it opens wait and go in order. A plain 404 upgrade
(an older Bops Cloud) sends every act by POST for the life of the
process; any other refusal or a socket failure sends that act by POST,
so the window still gets Bops Cloud's words, and so does the act after
Bops Cloud says one didn't go. Take and hand back stay on the POST. The
socket closes after 60 s without an act. scripts/test-browser-view.mjs
covers each rule against a fake Bops Cloud.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The viewer's acts go one at a time from an explicit queue. A move or a
scroll still waiting, with only moves and scrolls after it, is replaced
by the newer one of its kind (scrolls summed), so at most one of each
waits and moving never queues up ahead of a press; presses, releases,
keys and pages keep their order and are never dropped. MOVE_MS goes from
50 to 16 now that inputs reach Bops Cloud on one socket; the queue bounds
it on the POST path to an older Bops Cloud too. The opt-in probe's
queued, sent and settled events stay balanced when a move is replaced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The delay proxy reads the client's frames on WS /v1/browser/input and
logs each act as it logs a POST act (proxy_act_in, ready, connection,
out): act, method, mouse type and position only, never text. Without it
the probe's per-hop attribution can't find acts sent on the socket.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/verify browser-input runs the app server's sending test, Bops
Cloud's browser tests, both type checks and lint on the touched files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the socket it was answered 202 once sent and refused only at Bops
Cloud, so the window lost the 400 and its words (caught by
test-browser-tab-e2e criterion 4). It never reached the runner either
way; now it goes by POST and Bops Cloud answers it as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The person in control's press on the page draws the existing press ring
at that point right away (CursorFeed.ring), instead of never: the page's
own pointer event for it comes back as theirs (`you`) and is skipped.
Nothing changes in what reaches the page; a press that takes over still
goes nowhere and draws nothing. test-browser-tab-e2e criterion 4 checks
the ring within 400 ms, at the press point (it fails without this).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The input socket signed its key in once, at the upgrade. Its look every
ROLE_MS asked only mayUseBrowser, which for an owner's own bot reads
their state and never asks Orgo about the key again, and no member
notice comes for that case: a key revoked after the socket opened kept
driving the browser for as long as the socket lived. The look now asks
userForKey(key) first (cached as a POST's sign-in is) and ends the
socket (4403) when Orgo turns the key down or it is someone else's; a
5xx keeps it, as before. A revoked key now stops input within HIT_MS
plus ROLE_MS, the window a POST has. The watch stream has the same gap;
it is a separate follow-up.

Sol review round 1, S1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sam can drive as a Member of the owner's shared workspace, and up to three
others watch at once, each its own stream. The summary adds when each
watcher first painted the change and how far behind the driver it was,
and each watcher's frames a second while scrolling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cket does

An owner watching their own bot's browser was asked again every ROLE_MS
only about the owner's state, never about the key: a key Orgo turned
down kept the frames (the bot's signed-in pages) coming for as long as
the stream lived, since no member notice comes for an owner's own bot.
Both the stream and the input socket now ask through stillMay: the key
first (userForKey, kept as long as a POST's sign-in), then who may. A
turned-down key ends the stream with refused/signed_out in place of the
page; Orgo or the database out of reach still keeps it open.

Test: an owner's stream ends at the next look once the key is revoked,
and a new one is refused 401 (red without the change, green with it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Member's only tab is the browser, already open; pressing it closed it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…page at once

The runner caught a viewer up (latest frame, page, tabs, who has control)
only when a new person joined. Someone already watching who opened it
again, on another Mac or in another window, got a blank panel until the
page next changed. Found by the probe with two of the owner's windows.

Bops Cloud now says joined on the watch message when a stream opens, and
the runner catches up on it as for someone new. An older runner ignores
it (as today); an older Cloud never sends it.

Tests: Cloud tells the runner joined for a second stream of the same
person (red without, green with); the relay catches up on joined and not
on an unchanged list (red without, green with). Relay 15/15, Cloud
browser 16/16, e2e 17/17.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
setTimeout ended one of ten 6 s waits at 5999.9 ms by performance.now,
which failed the summary's own check after a whole run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread scripts/browser-speed/link.mjs Fixed
abhaypadmanabhan and others added 4 commits November 8, 2026 00:33
… host

CodeQL (js/request-forgery) on #183: the proxy built its outgoing URL with
new URL(req.url, target), so an absolute or scheme-relative request line
sent it to whatever host it named. It listens on 127.0.0.1 and runs only
for the probe, but there's no reason to allow it: the upstream host and
port are now fixed and only the path comes from the request; anything not
a plain path is a 400.

Test: an absolute and a // address are refused and never reach the other
server (red before: 200, forwarded), a path still goes upstream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s driving it at once

Before this branch each click or key was a POST that read the owner's
state, so a Member set to Read on a bot, hidden from it, or left without
Write on a bot newly sharing its computer was refused on the very next
act. On the input socket the access checked at the upgrade was kept until
the next ROLE_MS look (a minute). Per-bot levels and a bot's computer
live only in the owner's state, so no orgo-web notice comes for them. A
Member could keep clicking and typing in the bot's signed-in browser for
up to a minute, and after a bot moved to another computer, acts kept
going to the old computer's runner.

Now every write of the owner's state (their Mac, an Admin in People,
Bops Cloud's own) asks each open input socket and watch stream of that
account again, from the state as written: no database read, orgo-web's
answer as kept. A refusal ends it (4403, and the stream says why); a
changed computer ends it (4409). The hook sits in live.ts stateWritten,
so nothing that writes the state imports browser.ts. The ROLE_MS look
stays, for a turned-down key and orgo-web's changes.

Bops Cloud also pings each input socket every 15 s and drops one that
hasn't answered since the last ping. A Mac that changed networks leaves
its socket open with nobody there, and it was kept, and asked about every
minute, for good.

Tests (each red before, green after): a Member set to Read through PUT
/v1/state loses the socket and the stream at once (every is a minute); a
bot they don't have moved onto the computer ends a Member's socket, and
the bot moved to another computer ends the owner's (4409); a socket that
doesn't answer pings ends while one that does stays. The key-revoked
stream test now waits for its refused event instead of reading it as the
stream ends, which raced. Bops Cloud browser 16/16, core 369/369, edge
191/191.
…ps Cloud is asked again

A Bops Cloud from before the input socket (a plain 404 on the upgrade)
put every act on POST for as long as the Bops server ran. If the app
reached a Mac before Bops Cloud was deployed, that Mac stayed on the slow
path until Bops restarted. Now it's by POST for ten minutes, for that
Bops Cloud's address only, and then the socket is asked for again.

After a network change the Mac's socket could stay open with nobody on
the other end. Each act was answered 202 and went nowhere until a minute
passed with no act. The Mac now pings the socket every 5 s; one that
hasn't answered by the next ping is dropped, and the next act opens a
new one.

Tests (red before, green after): after a plain 404, acts go by POST
without asking again, another Bops Cloud still gets its socket, and once
the while is up the next act asks again and goes on the socket; a socket
whose Bops Cloud never answers pings is replaced, and one that answers
stays. scripts/test-browser-view.mjs 4/4.
@TigerYWang

Copy link
Copy Markdown
Contributor

Merged main (no conflicts) and fixed the review findings:

  • Who may drive, at each state write (d2735eb). The input socket kept the access it got at the upgrade until the next ROLE_MS look, but per-bot levels and a bot's computer live only in the owner's state, and orgo-web sends no notice for them. A Member set to Read, hidden, or left without Write on a bot now sharing the computer could keep clicking for up to a minute. After a bot moved, acts still went to the old computer's runner. Now every owner-state write (their Mac, an Admin in People, Bops Cloud's own) asks each open input socket and watch stream of that account again, from the blob as written, with no DB read. A refusal ends it with 4403 and the stream says why; a changed computer ends it with 4409. The hook is onStateWritten in live.ts, so state writers don't import browser.ts (that would make an import cycle that hits ROLE_MS before it's defined). The ROLE_MS look stays for keys and orgo-web.
  • Fallback to POST expires (fc3ce0c). A plain 404 now puts that Bops Cloud's address on POST for 10 minutes, then the socket is tried again. It no longer lasts for the server's whole life.
  • Heartbeat (both commits). The Mac pings its input socket every 5 s and drops one that didn't answer by the next ping, so after a network change acts stop going into a dead socket within about 10 s. Bops Cloud pings every 15 s and drops silent sockets, which it used to keep and look at every minute for good.
  • Test race: the key-revoked watch-stream test now waits for its refused event.

Each fix has a test that fails without it. Bops Cloud browser 16/16, core 369/369, edge 191/191; test-browser-view 4/4; relay 15/15; scripts/verify browser-input and browser-speed pass; both tsc runs and eslint are clean. The e2e passed 17/17 once; two other runs here failed only B2.2 (cursor 25.3 and 26.7 px a frame against 25), which is frame timing on this VM and not code touched here.

Not done: F4 (scripts/verify and FEATURES.md add/add with #186), which is left for #186's rebase.

@TigerYWang
TigerYWang merged commit 62dbf98 into main Nov 12, 2026
8 checks passed
TigerYWang added a commit that referenced this pull request Nov 12, 2026
Brings in #176, #178, #179, #183 and #185. Where they meet the new Your Apps:

- An Admin's review of an app whose production is on GitHub now reads Live from the
  last push, as the owner's own page does, not from Orgo's Published, which a push
  never moves. Bops Cloud passes the changeset last sent to GitHub with the repo and
  branch (SharedApp.github.at; null while it isn't linked, never the GitHub commit).
  From a Bops Cloud without it, Live is read from Published as before and Publish
  stays pressable.
- #179's words for where Publish goes are said in the Publish confirmation, not as a
  line under the row: "Publish sends it to GitHub (repo, branch)..." with the repo and
  branch as the destination, or "Visitors to <domain> won't see a publish until its DNS
  points at Orgo." with the domain. A domain that isn't live says its state in
  Technical details. #179's checks are ported to the new page.
- scripts/verify is one dispatcher for browser-speed, browser-input and your-apps
  (--list names them), and docs/FEATURES.md is one table.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants