Repository navigation
Pin the reference trajectory and add a UR5 expansion replay tutorial - #1244
Conversation
Order permission queries after the latest sensor sample on the caller's CUDA stream without resampling or advancing histories. Cover pending forbidden contacts and dropped evidence in standalone and history sampling paths. Validation: 75 targeted tests passed; Black 26.10.1, public API documentation coverage and agent context checks passed.
Share Default CPU numeric observations within bounded command-gather scopes, reconstruct immutable standard targets, and transfer fresh internal command ownership while preserving public snapshots and validation. Keep native multi-chunk grasp geometry and reductions resident on CUDA within each query, preserving samples, thresholds, stream ordering and independent results. Add focused regression coverage and a reproducible runtime comparison driver. Validation: 201 focused tests including CUDA and 42 context tests passed; Black 26.10.1, API docs 4924/4924 and diff checks passed. Sources match the final benchmark manifest. Forward/reverse sensor-free UR5 execution versus main 0119171 (seconds): small main 10.34-10.45, off 9.07-9.31, on 8.31-9.08; medium main 49.24-57.12, off 47.19-50.90, on 48.66-49.57; large main 251.77-260.14, off 189.57-196.42, on 192.07-200.29. All 492 program attempts accepted. Default CPU physics retains legacy CUDA grasp classification; these shared-host single-seed results are not physical qualification.
Restore the existing Default CUDA backend policy. Remove the newly introduced public selector and its ordinary-reporting test variant, retaining CPU contact permission coverage and documenting Direct GPU static-identity limits. Validation: 53 configuration/contact tests including CUDA and 42 context tests passed. Black 26.10.1, API docs and diff checks passed.
Preserve legacy catalog identities when phase paths are inactive, while keeping explicit phase declarations in registration fingerprints. Align preview error coverage with aggregate input support and update state test doubles for Direct GPU queries, including publication guard coverage. Validation: reproduced all 11 failed CI cases; 65 focused tests passed; broader Task Program and command/context suite 327 passed, 3 GPU cases skipped. Black 26.10.1, API docs 4963/4963, context map and diff checks passed. Existing context already describes inactive-default fingerprint compatibility; no context update required. Main synchronized through 438e111.
Move the existing pre-Scene collection after clearing facade-owned data and entity references. Cyclic wrappers newly orphaned by those clears must finalize while their Scene and World parents are still alive, instead of being collected during a later teardown. Extend the existing deferred-destruction test with a cyclic view finalizer; both registry-ownership cases fail before the change. Keep backend synchronization, ownership checks, physical validation and GC count unchanged. Validation: 288 manager CPU tests passed; the full 149-node failed CI GPU shard passed 143 with 6 pre-existing capability skips on pinned DexSim 0.6.0rc2. Black 26.10.1, API docs 4963/4963, context and diff checks passed. Lifecycle context already requires child-before-parent cleanup and remains accurate. The reported native abort itself was not deterministic locally; this fixes the demonstrated lifetime gap at its cleanup boundary.
…ings Identify explicit references after aggregate expansion; round contact holds up to the control grid and verify measured elapsed time; resolve the default-plane actor only when enabled. Retire expired warming attempts without proposal/credit and allow active peers to drain and export, including expiry during initial validation. Reject incompatible expert command modes, ActionManager declarations and controlled-joint selections before constructing a World. Preserve component-owned resource paths when relocating deployment snapshots by reusing the component resolver's resource normalization. Add focused regressions and extend real workload integration with an expired group alongside a peer that commits and exports a receipt. Validation: 106 expansion unit tests, 4 native integration tests on pinned DexSim 0.6.0rc2, and 42 context tests passed. Regressions reproduce all six review findings against 722e385. Black 26.10.1, API docs 4963/4963 and diff checks passed. Update workload guidance for action-schema compatibility and admission expiry.
Normalize ULP-scale error in exact-grid hold durations before ceiling. Measure hold elapsed time with bounded fsum reductions of actual physics callback intervals instead of subtracting accumulated World clocks. Preserve distinct samples, upward rounding of real off-grid durations, both-finger contact and drift requirements. Compare control_parts and active_joint_ids from resolved environment configurations rather than their shared implementation class. Replace artificial-class coverage with real decoded configurations and a full deployment-file regression sharing ordinary EmbodiedEnv. Validation: 117 expansion unit tests, four native integration tests on pinned DexSim 0.6.0rc2 and 42 context tests passed. Regressions reproduce all three follow-up findings on 8c244cd, including 50 Hz/100 Hz boundary holds, 500k/5M-step clock accumulation, truly short windows and pre-World schema rejection. Black 26.10.1, API docs 5010/5010, context map and diff checks passed. Update context for measured hold timing and configuration-owned joint selections.
Keep landing-pose acceptance independent of the raised release target, covering zero and default clearance as well as configured lowerer ownership. Reconstruct the coordinated-transport configuration through its normal validator instead of replacing its normalized binding dictionary. Validation: reproduced both failed CI cases; all 62 GenSim workflow tests and the native placement/release integration probe passed on pinned DexSim 0.6.0rc2. Black 26.10.1, API docs 5010/5010, context map and diff checks passed. No production behavior changed; existing GenSim context already describes clearance and landing validation correctly.
Bind the test validator to its measured environment and retain the real MeasuredTaskMonitor with deterministic joint evidence. This keeps hold-duration and contact assertions intact while exercising the new batched predicate boundary, instead of supplying an incomplete synthetic monitor. Validation: 471 related expansion/bridge/configuration/GenSim tests passed, plus both native validation/selected-handover probes on pinned DexSim 0.6.0rc2. Black 26.10.1, API docs 5013/5013, context map and diff checks passed. Dataset cache was isolated to task-owned tmpfs because the host root filesystem had no non-reserved free space. Production behavior and documentation contracts are unchanged.
Include scene preparation declarations and parameters in shared-World compatibility before creating the host. Reject cartesian_lift combined with phase_paths, consistent with Place's explicit conflict handling. Cover real deployment decoding, matching and mismatched preparation hooks, and public fixed-grid PickUp planning. Keep the dual-arm test robot aligned with selected-row FK queries. Update the affected project context.
Keep settling rows controller-active without trajectory or receipt credit. Retain configured grasp preload through coordinated placement. Use explicitly supported preceding-seed path kernels and cooperatively merge compatible requests, preserving active planning budgets and legacy solver fallbacks. Validation: actual reset/grasp negative regressions reproduce on the published SDK; selected-row real path batching passes, motion checks 123 passed, workload integration 3 passed, SSIK checks 2 passed. Five-task repeated physical qualification follows separately.
Keep W times B physical capacity and original attempt/time/byte/step limits. Competing retries reserve only running slots; pending writes and confirmed commits retain the hard quota. Cancel peers through the original command handshake after confirmed persistence, retaining canonical outcomes and measured partial evidence. Preserve phase metadata on safe stops and do not advance an already pending rejection hold twice. Native Task Program jobs enable idle-row reuse; custom sources opt in with an explicit cancellation port. Validation: 41 focused lifecycle, configuration and real simulation checks pass on DexSim rc2. The actual known-bad scheduler copy fails the new unexecuted-hold regression. Independent engine/robotics/verification passes completed; API/context/Black gates pass. Full five-task after measurements follow on an installed VCS package.
A finished runtime may still have its last ordinary motion awaiting env.step. Keep the default fail-closed abort contract, but allow the joint controller owner to supply actual applied target positions when no runtime safe hold exists. Never reuse the pending motion or acknowledge before real physics consumption. Retain the interrupted phase when terminal fields are empty, and keep the last actual scalar controller metadata and row identity in cancelled diagnostics without per-step serialization. Validation: 12 focused abort, safe-hold and actual reuse cases pass on rc2; full bottle trigger and repeated five-task qualification follow on the installed revision. Independent reviews completed.
Share simulator-free layout and feasibility declaration validation. Preserve final parent follower poses, distinguish sampled rejection from invalid authored anchors, retire exhausted warming rows, and record event transport history.
…codex/expansion-workloads
[High impact] The PR reaches replay users and multiple scene-screening workflows. The PR appears safe to merge, with a non-blocking fix needed for focused buttons when Space is pressed.
|
| if(event.target.isContentEditable || event.target.closest?.('select, textarea, input:not([type="range"]), summary'))return; | ||
| event.preventDefault(); | ||
| if(!event.repeat)$('play').click(); |
There was a problem hiding this comment.
Space activates the wrong button when a candidate or reset button has keyboard focus. The document-wide handler calls
preventDefault() and clicks the play button instead, so Space cannot activate the focused button and unexpectedly starts or pauses playback. Exclude focused buttons from this shortcut so they keep their normal keyboard behavior.
Prompt To Fix With AI
This is a comment left during a code review.
Path: embodichain/lab/expansion/_frontend/preview-controls.js
Line: 110-112
Comment:
**Space activates the wrong button** when a candidate or reset button has keyboard focus. The document-wide handler calls `preventDefault()` and clicks the play button instead, so Space cannot activate the focused button and unexpectedly starts or pauses playback. Exclude focused buttons from this shortcut so they keep their normal keyboard behavior.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Description
Keep the original recording visible while comparing how different layouts and grasps change a UR5 pick-and-place motion. This PR builds on #1192, which has merged into
main.What changed in ghost replay
On the target branch, clicking a candidate made that candidate the solid model. The reference and the other candidates had separate group visibility controls.
--referencenow stay solid throughout playback. If the user omits--reference, the first input recording supplies the solid model.Bilingual WebRTC replay
embodichain expansion previewnow opens the recorded replay over the native WebRTC viewport from #954. The language selector updates the replay controls, status messages, deviation curves and object heatmap in Simplified Chinese or English. Authenticated commands affect only recorded playback and the observer camera. Video and replay metadata travel separately.The preview uses a single browser transport. The former JPEG snapshot routes, image readback and encoding, and transport flags are removed.
embodichain.lab.expansion._frontend. The installed wheel served the bilingual UR5 WebRTC replay and generated an offline distribution report. 99 focused and context tests passed. Black and API documentation checks passed.Scene-first collision and layout screening
scene-first-checks.jsonundermesh_convex_parts. Physical execution and measured qualification use the original colliders.Eight-environment verification
The downstream CLI ran these profiles using the installed EmbodiChain wheel at
6e53943bd934f0c0555f800ce3514c8e536f837cand DexSim0.6.0in Python 3.12. The visualization branch includesmainatb715f35d2.In the three scene-first columns, each cell gives accepted episodes / candidate attempts. Each task ran in one World with eight environments. All 27 accepted episodes passed the task, event, terminal-state and native path-contact checks. The CLI exported them to LeRobot and read them back. The saved-run integrity checks verified 1,672 artifacts. Each browser served the recorded frames, deviation curves and three initial object-pose samples with zero physics steps.
The standard preset column gives accepted / requested episodes. Each standard preset collected one episode per operator, including via points and null-space residuals, for a separate total of 15 accepted LeRobot readbacks. Direct reads of the Parquet files verified 56,087 frames across both presets, with contiguous indices, increasing timestamps and finite measured joint states and actions.
The bundled workspace pick-and-place integration test also recorded eight accepted episodes for each of
scene_first,robot_firstandbidirectional.Run the downstream scene-first examples
Start in the DexEChain checkout containing the three task profiles. Prepare the task assets listed in its Sim2Real guide and use a package source that provides DexSim 0.6.0. The downstream simulation extra and runtime check share the exact upstream revision. If an older shell session exported a revision override, clear it with
unset DEXECHAIN_SIM2REAL_EMBODICHAIN_REVISION.Each command opens the replay after exporting nine episodes. The first recording supplies the solid comparison. To supply a separately recorded original, use the tutorial below. The Piper profile samples translations up to ±2 mm along the bookshelf X and Y axes and checks a 2.8 s native two-finger hold.
Piper trajectory perturbation defaults
The downstream Piper presets use
trajectory.joint_offset_scale: 0.006, twenty times the previous0.0003. The nativejoint_residual,via_pointsandnullspace_residualoperators share this parameter. It sets the sampling amplitude to 0.6% of each controlled joint’s travel. The active operator list still comes from each preset.In the amplitude sweep at
6530d9cd0with DexSim0.6.0rc2, each setting used the same three book layouts, candidate seed 2 and nominal seed 1234. Each perturbed rollout had the same starting state and original right-arm approach commands as its unperturbed control. TCP poses were calculated from measured joint positions and the configured tool transform.Each cell is the mean of three per-episode peaks during the 4.52 s
pick_up.approachphase. The joint metric takes the largest absolute deviation among the six right-arm joints at each frame. All 12 perturbed recordings and three controls passed physical acceptance and LeRobot readback. The standard five-operator Piper preset also recorded one accepted episode per operator, including a null-space trajectory within the configured TCP translation and rotation tolerances.Run the UR5 example
Run these commands from the repository root with EmbodiChain, DexSim and LeRobot installed in the
py312environment. Use a fresh output directory for collection. If recordings already exist, skip the three collection commands and point--referenceat the job directory containing the original episode.scene_firstproposes tabletop layouts and checks that the arm can execute them.robot_firststarts from reachable arm poses and derives compatible layouts. Each strategy collects four fixed-graspobject_poseepisodes and fouraffordanceepisodes for the pick, yielding 16 augmentations alongside one original. The tutorial defaults to seed 7. Tutorial README · Expansion configurationType of change
Validation
After merging
main, 174 focused tests passed with GPU tests enabled, including task discovery from an installed downstream package, wheel resources, contact sensing, expansion screening and replay. API documentation coverage was 5,132/5,132. The task catalog uses the existing environment-component path resolver, so the documented configuration contract still applies.165 focused scene expansion, feasibility and simulation-adapter tests passed with GPU tests enabled. Another 18 acceptance and layout-collection tests and 42 project-context tests passed. The regression cases check an open passage in a volumetric mesh, physical-ID permissions across parts in scalar and batched checks, bounded shelf overhang, rejection over a gap and persisted layout validation. The 514-shape test exercises 65,792 eligible native pairs and selected-row ordering across chunks.
Sixteen probe points checked the two occupied Piper shelf slots, 10 to 80 mm above each book base. The previous bounding box covered all 16. The original triangle surface and the new convex parts both classified all 16 as free. A separate planar annulus retained its open center and material on both sides.
The installed wheel at
6e53943bdcontains both the replay JavaScript files and the bundled web assets frommain. The downstream wheel contains all three scene-first profiles, with byte-identical source and packaged YAML. The integrity checks and browser readbacks are reported above.On an AMD Ryzen 9 9950X with one CPU thread and Newton 1.7.0, the median of 20 warmed scalar queries was 0.49 ms at 33 shapes on both revisions. The 514-shape case completed in 2.89 ms with about 28 MiB of additional process peak RSS, using a 64-contact buffer. The previous revision rejected that scene at construction.
22 preview, preview-job and measured comparison tests passed, along with 42 project-context tests. Heatmap assertions cover unequal X and Y spans, sub-millimeter variation, constant positions, and counts in the outermost bins.
Verified all 2,048 displayed cell counts against the eight original
object_poseepisode files. Desktop hover and mobile taps read back counts at the edges. Tested object selection, ghost filtering, the affordance-only empty state and distinct tick labels at narrow ranges. Native replay reported zero physics steps.Built the wheel from
f3bcfb24eand verified the packaged viewer HTML, both JavaScript assets and the Python heatmap and loader modules against the source files.Checklist
black .with Black 26.10.1.python docs/scripts/check_api_docs.pysuccessfully.Follow-up #1251 measures and reduces grasp retries in the three eight-environment examples.