feat: M4/M5 features (cloud sync, share links, Yjs CRDT, M5 validators + board) + M0–M3 audit#1
Conversation
End-to-end BOM pricing reaches the React surface:
kiserver
- New `_bom_lines_from_project` walks `project.pcb.footprints`,
groups by mpn with summed qty, skips empties / missings /
whitespace-only mpns.
- New `GET /project/{id}/bom/price?qty_multiplier=N&force_refresh=B`
runs the M3-P-05 `build_default_aggregator().price_bom()` over
the scaled list and returns `{ok, project_id, bom_lines,
pricing: {parts, distributor_totals_usd, grand_total_usd,
missing_mpns, errors}}`. Aclose-on-exception so a mid-fan-out
failure doesn't leak the SQLite cache handle.
client
- `BomView` panel: auto-fetches on mount + on every qty change,
Refresh button forces the cache bypass. Header shows line count
/ placement count / grand total / cart-split. Per-MPN table with
refdes count / qty / clickable distributor link
(target=_blank + noopener+noreferrer) / unit $ / line $ / stock /
colour-coded lifecycle (active=green / nrnd=amber /
obsolete=red / preview=blue). Warn rows surface missing MPNs +
per-distributor errors. Qty input clamps to >= 1.
Tests
- 9 Python: `_bom_lines_from_project` corner cases (grouping +
skip empties + missing pcb/footprints/keys), endpoint
round-trip, qty_multiplier scaling, force_refresh propagation,
unknown project_id 404, 0-qty rejection, aclose-on-exception
via `raise_server_exceptions=False` TestClient.
- 10 React: empty / fetch on mount / distributor link contract /
Refresh / qty change re-fetch / qty clamp / server error
preserves rows / missing-MPNs warn / distributor-errors warn /
empty-board hint.
Risk flags (single source, low stock, NRND) land as a follow-up
once Mouser + Octopart adapters give "single source" real meaning.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…2, P-04)
Three more real distributor adapters plug into the M3-P-05
`DistributorAdapter` ABC + aggregator — no aggregator changes
needed beyond autoload wiring. Every one is real HTTP against
documented APIs, tested via httpx.MockTransport, and raises
`DistributorAuthError` (not fake data) when credentials are absent.
M3-P-02 Mouser (Search API v2):
- `?apiKey=` query-param auth (no OAuth). partnumber search →
keyword fallback. Culture-formatted price parsing ("$1.65",
"€1,42", "1,234.56") with US/EU separator heuristic. Lifecycle
normalisation. 200-with-Errors[] envelope mapped to auth vs
transport. 16 tests.
M3-P-01 Octopart (Nexar GraphQL):
- OAuth client_credentials mint at identity.nexar.com with
expiry-aware token cache + concurrent-mint dedup. One
`supSearchMpn` query expands every seller×offer into a PartQuote
(cross-distributor view from a single call); sellers surface as
`octopart-via-<seller>`. Prefers `convertedPrice` for USD-to-USD
comparison. GraphQL-envelope auth errors clear the token cache.
13 tests.
M3-P-04 JLCPCB (SMT component list):
- Key-optional (anonymous quota works; JLCPCB_SESSION_COOKIE
raises it). CNY→USD via cached er-api.com daily rate with
JLCPCB_CNY_TO_USD env override. Surfaces jlc_assembly_basic /
jlc_assembly_extended flags from componentLibraryType. Handles
both componentInfoList + componentPageInfo.list envelope shapes.
12 tests.
`build_default_aggregator()` now autoloads Mouser (MOUSER_API_KEY),
Octopart (OCTOPART_CLIENT_ID+SECRET), and JLCPCB (unconditional —
anonymous quota) alongside Digi-Key, each with an `include_*=False`
opt-out for isolation. 5 new factory tests cover the autoload
matrix + the full-house case.
Full distributors + dependent suite: 179 passed (82 distributor +
PCB tools + BOM endpoint). Ruff clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…03, 1/N)
First checkpoint of the multi-session push-and-shove router port.
This lands the two foundational layers the shove algorithm builds on;
the recursive shove + head-advance loop + wasm/UI wiring follow in
subsequent sessions.
v1 scope pinned in the module docs: straight segments only (no arcs),
vias + pads are fixed walls (no movable-via shove), per-layer
collisions, walk-around fall-through when budget exhausts.
routing/shove/geom.rs — the shove atom:
- Vec2 displacement type + translate/normalized/scaled helpers.
- closest_on_segment, segment_segment_closest/_distance,
segment_intersection.
- push_vector(head, obstacle, clearance) → the minimum perpendicular
displacement to apply to an obstacle so the two thick centerlines
clear the required distance. Returns None when already clear;
falls back to the head's left-normal when closest points coincide
(crossing). 13 unit tests incl. the "applied push actually reaches
clearance" round-trip + parallel-overlap / perpendicular-T /
crossing / exact-clearance / no-collision cases.
routing/shove/world.rs — the obstacle model:
- ShoveItem enum (Track{locked} / Via / Pad) with is_shovable
(only unlocked tracks), on_layer, half_extent_mm.
- ShoveWorld.collisions_with(head, layer, net, ignore) → per-layer,
own-net-excluded collision list. Each Collision carries the
shovable flag, center distance, required distance, penetration,
and the obstacle segment index for tracks.
- The `ignore: &HashSet<ItemId>` cycle-detection hook is already
honoured by the query so the recursive shove slots in without
touching the world API (per advisor guidance). 12 tests.
routing/shove/mod.rs — ShoveBudget with INDEPENDENT
max_recursion_depth + max_total_shoves bounds (advisor: a wide-but-
shallow shove and a deep-but-narrow one are capped separately).
cad lib: 180 tests (155 prior + 25 shove). Clippy + fmt clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…(M3-R-03, 2/N)
Builds on the geom+world foundation with the actual shove algorithm
and the routable entry point.
routing/shove/engine.rs — recursive shove:
- shove_head(world, head, layer, net, budget) → ShoveOutcome
{Resolved | BlockedByWall | Cycle | BudgetExhausted}. Clone-try-
commit transaction: the caller's world only changes on Resolved,
so a failed shove leaves the board pristine for walk-around
fall-through.
- Top-level loop queries head collisions, shoves the first shovable
one via push_vector, then RECURSES — the moved obstacle becomes a
new head clearing its own neighbours. Walls (via/pad/locked) →
BlockedByWall. Explicit cycle detection via the path set (shoving
an item already up-stack → Cycle). Dual independent budgets
(recursion depth + total shoves) enforced separately.
- 10 tests: no-collision / single-shove / wall block / locked block /
rollback-on-failure / 3-track cascade / both budget-exhaustion
paths / own-net + cross-layer exclusion.
routing/shove/route.rs — head-advance entry point:
- route_shove(world, ShoveRouteInput) → ShoveRouteResult
{Routed{route_mm, moved[], shoves_applied} | FellBack{reason} |
InvalidInput}. Diffs pre/post world to report exactly which tracks
moved (with new geometry) so the caller persists them. FellBack
is the documented walk-around-fall-through signal.
- ShoveBudget gains serde derives (default 8 depth / 64 total) so it
round-trips the wasm boundary; budget JSON round-trip test pins it.
- 10 tests incl. clear-lane / shove-and-report / wall fall-back /
budget fall-back / invalid-input / non-mutation.
world.rs gains segment_clearance_ok (head-vs-obstacle DRC check used
by the engine tests + the next session's multi-segment head walk).
wasm.rs: routeShove(request_json) bridge — `{world, input}` →
ShoveRouteResult JSON — so the M3-T-05 RouteTool gesture can run PnS
live and fall back to the M2 walkaround `route` shim on FellBack.
cad lib: 199 tests (44 shove). Clippy + fmt clean; wasm pkg rebuilt
with the routeShove export.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…-R-03 done) useShoveRoute drives the M3-R-03 shove router live on a start→cursor drag: builds the ShoveWorld from project tracks/vias on the active layer, surfaces the proposed route + shoved-track ghosts as `plan`, flags `fellBack` so the caller defers to the M2 walk-around router, and commits the new + moved tracks through the new `ui_shove_apply` gateway tool (mapping each shoved synthetic ItemId back to its project uuid). Also sorts two stray `__all__` lists to clear the ruff RUF022 gate. Closes M3-T-05; marks M3-R-03 v1 (straight-segment shove) complete. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three structural fixes cut the zone-fill XOR error vs KiCad's reference 15-115x and bring all three golden fixtures under the 0.01mm Hausdorff target (concave + thermal also under the 0.01mm² XOR target): - Same-net pad keepout is the spoke CROSS, not the full pad rectangle. The keepout is now the full `pad ⊕ gap`; spokes (inner endpoint at the pad centre) carve the 4-arm cross KiCad fills. simple 0.214->0.017, thermal 0.809->0.094. - Spokes are unioned back at full bridge_width AFTER the min-thickness opening instead of cut before it, so the opening's erode can't narrow them at the drill edge. thermal 0.094->0.045 (Hausdorff 0.094->0.014). - Circle + rounded-rect keepouts use KiCad's GetArcToSegmentCount + ERROR_OUTSIDE radius correction + half-step phase (reverse-engineered from the reference fixtures). thermal 0.045->0.007, simple 0.017->0.0144. Final: concave 0.0025/0.0050, simple 0.0144/0.0091, thermal 0.0070/0.0055 (XOR mm² / Hausdorff mm). Gate tightened ~50x/40x to 0.02/0.01; the spec's 0.01mm² XOR target stays open for `simple` under the new M2-R-05d follow-up (min-thickness opening's offset arc decomposition). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… (M2-R-05c) The opening's round-corner offset used a fixed 64-segment disc; KiCad's SHAPE_POLY_SET::Inflate/Deflate tessellates round joins to m_MaxError (~12 segments for the 0.125mm opening radius). Deriving disc_polygon's segment count from m_MaxError (inscribed, phase-0, Clipper jtRound-style — NOT the keepout's circumscribing ERROR_OUTSIDE form) drops the zone-fill XOR on all three golden fixtures: concave 0.0025->0.0010, simple 0.0144->0.0123, thermal 0.0070->0.0047. simple's remaining 0.0123 (vs the spec 0.01 target) is the phase mismatch between our fixed-phase Minkowski disc-union and KiCad's edge-aligned round joins, compounded at corners — needs an edge-aligned offsetter (M2-R-05d). Gate tightened XOR 0.02->0.015 (66x tighter than the original 1.0); checkbox stays open until simple hits 0.01. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ible in f64 Records the full investigation into closing the simple fixture's zone-fill XOR from 0.0123 to the spec's 0.01 mm². Six independent approaches were implemented and measured, all reverted: - edge-aligned round-join sector (f64): spikes at reflex vertices - phase-aligned full disc: no-op - integer-coordinate boolean kernel: no-op (i_overlay already snaps to a fine internal integer grid) - faithful Clipper2 ClipperOffset port (debugged through 5 real bugs to a working state — concave IMPROVED to 0.000294): nets worse, removes a thin inter-pad neck KiCad keeps (knife-edge min_thickness decision) - clipper2 FFI: architecturally blocked — crates/cad compiles to wasm, C++ can't target wasm32-unknown-unknown All converge on the f64-vs-integer-nanometre precision floor: matching KiCad to 0.01 requires reproducing its exact integer-nm geometry pipeline end-to-end, a multi-week from-scratch engine. The 0.015/0.01 gate stays; the structural fidelity (spoke cross, full-width spokes, KiCad keepout arcs) committed in M2-R-05c is correct. Includes the multi-agent web-research report on Clipper2/KiCad offset internals and the Rust polygon-offset crate landscape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…pening sequence Built and tested the dedicated integer-nm ClipperOffset engine (rings snapped to 1nm, i_overlay integer Overlay cleanup). It produced byte-identical numbers to the f64 ClipperOffset port (concave 0.000294, simple 0.032684, thermal 0.036609) — conclusively ruling OUT coordinate precision as the cause of the inter-pad-neck regression. The neck removal is algorithmic: KiCad's exact min-width opening SEQUENCE (deflate/inflate order), not a finer grid. Reaching simple ≤0.01 needs reimplementing KiCad's ZONE_FILLER min-width pass, a multi-week effort, not a precision tweak. Baseline (0.0123, 0.015 gate) retained; code unchanged from HEAD. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements the five user-directed M4/M5 items plus the audit that preceded them. Everything is tested; details in Todo.md. Audit + fixes: - docs: Todo.md — evidence-based M0–M3 gap audit - docs(plan): correct M3-P-07/M3-C-08 subagent paths to subagents/__init__.py KCIR (schema bump 0.4 -> 0.5): - feat(kcir): add pcb.signoff (rf/ddr/bga_fanout_reviewed) + migration v0_5; regenerate ts-rs bindings + client store type M5 validators + reference board: - feat(validate): KC060 (DDR fly-by) + KC070 (BGA fanout) validators - feat(examples): esp32_c6_rf — 4-layer RF + DDR3L-BGA co-pilot board - feat(ki): canonicalize dev tool for authoring round-trip-safe examples FR-007 cloud sync: - feat(kiserver): content-addressed object store (LocalFs + lazy-boto3 S3) - feat(kiserver): cloud sync push/pull routes FR-080 share links: - feat(kiserver): read-only share routes (content-addressed token) - feat(client): #/share/<token> SharePage (read-only kicanvas preview) FR-081 multiplayer (P4 -> Yjs, ADR-0001): - feat(server): Yjs CRDT relay + WS /crdt/:projectId (off by default) - feat(client): CrdtSession + WS transport - feat(agent): PreToolUse hard-gate so Claude cannot flip pcb.signoff.* Chores: - chore(deps): add yjs (server, client); boto3 [s3] extra + moto (kiserver dev) - chore: remove stale committed ts-rs export tree crates/client/; .gitignore guard Tests: cargo test --workspace green; Python 466 passed; client 478; server 25. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolves 19 ruff findings that predate this branch (in 9 files left untouched by the M4/M5 work) so the PR's Python gate runs green: - I001 import-block ordering (×8): test_activity, test_ask_user, test_subagents, kiconnector/main, dfm, test_dfm, diff_cli - RUF022 sort `__all__` in kiserver/dfm.py - F401 drop unused imports (test_build_cli: typing.Any; test_diff_cli: asyncio, PcbSummary) - E501 wrap long lines (build_cli argparser help, test_build_cli, test_diff_cli signatures/calls) - RUF003 ambiguous en-dash → hyphen in a build_cli comment Behaviour-preserving: import order, formatting, and dead-import removal only. Full Python suite stays green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Sorry @LayerDynamics, your pull request is larger than the review limit of 150000 diff characters
There was a problem hiding this comment.
Code Review
This pull request introduces several major features and enhancements across the codebase, including the implementation of the M3-R-03 push-and-shove router in Rust (compiled to WebAssembly), the M3-T-08 live distributor-priced BOM view, FR-080 read-only share links, and FR-081 Yjs-based real-time multiplayer sync. It also pulls forward M5 capabilities, adding design sign-off gates and the esp32_c6_rf reference project. The review feedback highlights critical improvement opportunities, such as resolving a room-release race condition in the CRDT hub, adding pre-check validation to prevent corrupted file restores, optimizing BGA pad pitch calculations to avoid blocking the main thread, fixing European currency parsing, and guarding against concurrent fetch race conditions in the BOM view.
| export class CrdtHub { | ||
| private readonly rooms = new Map<string, CrdtRoom>(); | ||
|
|
||
| room(id: string): CrdtRoom { | ||
| let room = this.rooms.get(id); | ||
| if (!room) { | ||
| room = new CrdtRoom(id); | ||
| this.rooms.set(id, room); | ||
| } | ||
| return room; | ||
| } | ||
|
|
||
| /** Drop a room once empty so an idle project doesn't pin its doc. */ | ||
| release(id: string): void { | ||
| const room = this.rooms.get(id); | ||
| if (room && room.connectionCount === 0) { | ||
| this.rooms.delete(id); | ||
| } | ||
| } |
There was a problem hiding this comment.
There is a subtle race condition in CrdtHub where a room can be deleted from the hub while a new connection is upgraded but has not yet joined the room (which happens in onOpen). This splits clients into different room instances for the same project ID. We can completely eliminate this race condition by implementing a reference-counting mechanism that tracks upgraded-but-not-yet-opened connections.
export class CrdtHub {
private readonly rooms = new Map<string, CrdtRoom>();
private readonly refs = new Map<string, number>();
room(id: string): CrdtRoom {
let room = this.rooms.get(id);
if (!room) {
room = new CrdtRoom(id);
this.rooms.set(id, room);
}
this.refs.set(id, (this.refs.get(id) ?? 0) + 1);
return room;
}
/** Drop a room once empty so an idle project doesn't pin its doc. */
release(id: string): void {
const count = (this.refs.get(id) ?? 0) - 1;
if (count <= 0) {
this.refs.delete(id);
this.rooms.delete(id);
} else {
this.refs.set(id, count);
}
}|
|
||
| // Browser-safe base64 (no Node Buffer). btoa/atob exist in every target | ||
| // browser and in the happy-dom test env. | ||
| export function encodeU8(u8: Uint8Array): string { | ||
| let s = ""; |
There was a problem hiding this comment.
Using string concatenation in a loop to encode a Uint8Array to a binary string can block the main thread for a significant amount of time on large Yjs updates. We can optimize this by processing the array in chunks using String.fromCharCode.apply, which is orders of magnitude faster and avoids call stack limits.
export function encodeU8(u8: Uint8Array): string {
const len = u8.length;
let s = "";
const chunk = 16384;
for (let i = 0; i < len; i += chunk) {
s += String.fromCharCode.apply(
null,
u8.subarray(i, Math.min(i + chunk, len)) as unknown as number[]
);
}
return btoa(s);
}| const load = useCallback( | ||
| async (force: boolean) => { | ||
| setLoading(true); | ||
| setError(null); | ||
| try { | ||
| const params = new URLSearchParams(); | ||
| params.set("qty_multiplier", String(qtyMultiplier)); | ||
| if (force) params.set("force_refresh", "true"); | ||
| const url = `${apiBase}/project/${encodeURIComponent(projectId)}/bom/price?${params}`; | ||
| const resp = await fetchImpl(url, { method: "GET" }); | ||
| const body = (await resp.json()) as BomResponse; | ||
| if (!resp.ok || body.ok === false) { | ||
| throw new Error(body.error ?? body.detail ?? `${resp.status} ${resp.statusText}`); | ||
| } | ||
| setLines(body.bom_lines ?? []); | ||
| setPricing(body.pricing ?? null); | ||
| } catch (err) { | ||
| setError(err instanceof Error ? err.message : String(err)); | ||
| } finally { | ||
| setLoading(false); | ||
| } |
There was a problem hiding this comment.
Since load is an asynchronous function, if multiple requests are in flight concurrently (e.g., when the user quickly changes the quantity multiplier), a slower earlier request might resolve after a faster newer request, overwriting the state with stale data. We can prevent this race condition by using a reference to track the latest request ID and ignoring stale responses.
async (force: boolean) => {
const requestId = ++lastRequestId.current;
setLoading(true);
setError(null);
try {
const params = new URLSearchParams();
params.set("qty_multiplier", String(qtyMultiplier));
if (force) params.set("force_refresh", "true");
const url = `${apiBase}/project/${encodeURIComponent(projectId)}/bom/price?${params}`;
const resp = await fetchImpl(url, { method: "GET" });
const body = (await resp.json()) as BomResponse;
if (requestId !== lastRequestId.current) return;
if (!resp.ok || body.ok === false) {
throw new Error(body.error ?? body.detail ?? `${resp.status} ${resp.statusText}`);
}
setLines(body.bom_lines ?? []);
setPricing(body.pricing ?? null);
} catch (err) {
if (requestId === lastRequestId.current) {
setError(err instanceof Error ? err.message : String(err));
}
} finally {
if (requestId === lastRequestId.current) {
setLoading(false);
}
}
},
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
feat: M4/M5 features (cloud sync, share links, Yjs CRDT, M5 validators + board) + M0–M3 audit
Summary
Two threads of work:
Todo.md(evidence-based gap analysis), plus a fix to theM3-P-07/M3-C-08plan entries (they named a non-existentagent/agents/path; the subagents actually live inagent/subagents/__init__.pyand are wired throughbridge.py).What landed
pcb.signoff(rf/ddr/bga_fanout_reviewed) + migration 0.4 → 0.5 (v0_5.rs), ts-rs regen, client store typekc_validateexamples/esp32_c6_rf/— real 4-layer board (ESP32-C6 + DDR3L BGA 4×4 ball grid + U.FL RF + GND plane + CPWG feed); round-trips byte-identical, opens viaKiProject::open. Newcargo run --example canonicalizeauthoring toolLocalFsdefault + lazy-boto3S3, env-selected) +sync.py+POST /project/{id}/sync/push,POST /sync/pullPOST /project/{id}/share,GET /share/{token},GET /share/{token}/file+ client#/share/<token>SharePagecrdt.tsroom +WS /crdt/:projectId, off by default behindKICLAUDE_MULTIPLAYER) + clientCrdtSession. P4 decided → Yjs (docs/ADR/0001-crdt-yjs.md)PreToolUsehard-gate denies any Claude attempt to flippcb.signoff.*, even in trusted mode (SPEC §11 M5: "the LLM cannot flip sign-off")A second commit (
chore:) fixes 19 pre-existing ruff findings in 9 untouched files so this PR's Python gate runs green (behaviour-preserving: import order / formatting / dead-import removal only).Test evidence
cargo test --workspacegreen (cad 199, ki 162 + golden round-trip suites);cargo fmt --check --all+ clippy clean.ruff check services/clean.KCIR_VERSIONbumped +v0_5migration entry added — policy satisfied.Scope / known limits
WS /crdtroute glue follows the same untested-by-unit-tests pattern as the existing/ws; the room logic is unit-tested.yjs(server, client);boto3([s3]extra) +moto(dev) in kiserver.Todo.md§1–§5 remain open (not part of this request).🤖 Generated with Claude Code