feat(webapp): let the hub hold the document - #241
Merged
Merged
Conversation
collab.go is a relay: it never parses a frame, so nobody owns the document, and every property that needs an owner is faked somewhere else. A seed CLAIM with a grace timer, because two clients seeding one file build two documents that duplicate every character on merge. A byte cap on a log that only grows, and a rebuild-from-scratch when it is hit. A client-side snapshot rule — "whoever stops typing last writes the file" — that makes N co-editors write N versions of identical text. And a failure mode with teeth: a client that loses the relay edits its own buffer, so two browsers hold two documents and overwrite each other. That cost a user six characters (#234), which is why upload/content now takes If-Match and parks the loser as a conflict copy. The copy is the right safety net; needing one every time a laptop changes network is not. So the hub holds it. github.com/reearth/ygo v1.50.0: pure Go, no cgo — which is the constraint that made this a relay in the first place and the one that has since changed — mounted as an http.Handler behind the same proj() wrapper every other per-project route uses. The decision this rests on is recorded in the PRD and is not mine: until now the hub never parsed a client-supplied CRDT update. It does now, deliberately, with the conditions that came with it. Three things the handler gets right, each with a test: - THE ROOM NAME IS THE HUB'S. ygo reads it from PathValue("room") or the URL's last segment, so a caller who could name the room would make the project id in the path decoration — any member of any project could join any other project's document by asking for its name. The handler derives (project, path) itself, after proj() has resolved the project. Verified failing without it. - PermRead, not PermWrite. A read-only member may OPEN a file and watch it being edited; their connection is marked read-only and their writes are dropped server-side, which is what read-only means everywhere else here. - A path the caller cannot see is 404, never 403, because a 403 confirms the file is there. Beside the relay rather than instead of it, for one release. The wire fixtures are now a CI test. They were produced by the exact yjs build the frontend ships and are checked in as BYTES, because CI runs Go without node and a test that needs a toolchain it does not have is a test that gets skipped. Both directions, both encodings, including a 10,000-op document. A version bump that breaks the wire fails the build instead of the editor. Stage 0 and Stage 1 of docs/collab-provider-prd.md. Memory measured at ~100 KB per actively-edited document (every keystroke is its own item until GC merges them), two orders of magnitude under the 8 MiB log cap it replaces — so the limit worth having is eviction, not bytes. Go: ok, 412s, 0 failures. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The seed CLAIM existed because a relay cannot seed: it never parses a frame, so it has no way to turn a file into a document. Exactly one joiner was therefore told "you are first, build it from the bytes you loaded" — with a grace timer, because a claim that never produced anything left every later joiner holding a blank document that the source editor would then cheerfully save over the file. A hub that holds the document can just do it. LoadDoc runs once, when the room is created and before any client is attached, so there is nothing to claim and nothing to race: the document starts as the file, deterministically, every time. A file that does not exist yet starts empty, which is an ordinary thing to open an editor on and never an error. Bounded by maxSeedBytes, because editing costs roughly ten times the content in CRDT items — a memory ceiling rather than a file-size opinion. StoreUpdate is left returning nil deliberately, and says so: snapshotting the document back to the file is Stage 3, and until then the client still saves through upload/content exactly as it does now. Completes Stage 1 of docs/collab-provider-prd.md, including the "rebuilt from the file deterministically" criterion that the previous commit claimed the stage without. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every per-project route the hub serves has to be classified in desktopRoutes, because one the desktop app does not know is answered from LOCAL STATE — a plausible wrong answer rather than an error, which the comment in that file says has shipped twice. /ycollab has a sharper consequence than the relay it sits beside: the document IS hub state now, so a desktop answering locally would hand the editor a second, private document, and the first thing that document does is get saved over the file. streaming() learns about websockets while here. It decides which client serves a request BEFORE the answer exists, by asking whether the caller wants a long-lived connection — an upgrade is exactly that, even though it is not a stream of frames the proxy can read. Caught by CI, not locally, because I ran ./internal/webapp and CI runs ./... — the test that fails lives in cmd/bdrive. Full module now: 12 packages ok. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…over A websocket handshake ends with the handler HIJACKING the connection. The gzip wrapper added in #232 implements Flush and Unwrap but not http.Hijacker, so the upgrade could not happen and the client got a 500 it had no way to explain. Invisible until now because nothing in this hub upgraded anything. It became visible the moment a browser tried to reach the hub-held document — and only from a browser: Go's websocket dialer sends no Accept-Encoding, so the middleware never wrapped it and the Go test passed while every real client failed. The test now dials the way a browser does, and fails without the fix with exactly the 500 that was observed. Skipped rather than made hijackable. There is nothing to gzip on a handshake, and a Hijack method on a compressing writer is a trapdoor that returns a socket somebody may already have written a gzip header to. This is the second capability that wrapper has had to learn to forward — Flush was the first (#234, the unflushable-writer hang). Wrapping a ResponseWriter means answering for every interface the real one implements, and getting it wrong reads as an unexplainable 500 or a hang rather than a compile error. Also here, because the route is not reachable without them: the {room...} variant y-websocket's URL produces, and both ycollab routes classified for the desktop app — an unclassified route is answered from LOCAL state, which for a document the hub now owns would hand the editor a second, private copy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
reearth/ygov1.50.0, pure Go, no cgo, mounted behind the sameproj()wrapper as every other per-project route.yjsis now a CI test, checked in as bytes so CI needs no node.What "nobody owns it" costs today
maxRoomBytes(8 MiB) + rebuildThat last one cost a user six characters (#234). The conflict copy is the right safety net; needing one every time a laptop changes network is not.
The decision this rests on
Until now the hub never parsed a client-supplied CRDT update —
collab.gostores opaque bytes and hands them on. That ends here, deliberately, recorded in the PRD with the conditions it carried: behindproj(), bounded inputs, old relay reachable for a release.The constraint that originally made this a relay — "a cgo y-crdt would break the cross-compiled release the same way a cgo sqlite would" — no longer holds.
Three things the handler gets right
ygoreads it fromPathValue("room")or the URL's last segment. A caller who could name the room would make the project id in the path decoration — any member of any project could join any other project's document by asking for its name. Verified failing without the guard:room = "../some-other-project/secrets.md".PermRead, notPermWrite. A read-only member may open a file and watch it being edited; the connection is marked read-only and their writes are dropped server-side. That's what read-only means everywhere else here — the relay'sPermWritewas the coarser answer.Wire fixtures, in CI
Produced by the exact
yjsbuild the frontend ships, checked in as bytes — CI runs Go without node, and a test that needs a toolchain it doesn't have is a test that gets skipped.Memory, measured
~100 KB per actively-edited document — every keystroke is its own item until GC merges them, so editing costs ~10× the content. A hub with 100 open documents is ~10 MB, two orders of magnitude under the 8 MiB-per-room log cap this replaces. The limit worth having is therefore eviction, not bytes.
Verification
Go: ok, 412s, 0 failures,
vetclean. Stages 0 and 1 ofdocs/collab-provider-prd.md.🤖 Generated with Claude Code