Skip to content

Standardize native expansion collection, acceptance and previews - #1192

Merged
yuecideng merged 49 commits into
mainfrom
codex/expansion-workloads
Nov 19, 2026
Merged

yuecideng merged 49 commits into
mainfrom
codex/expansion-workloads

Conversation

@yuecideng

@yuecideng yuecideng commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Description

Provide a native collection path so downstream Task Program tasks only declare their scene proposals, plan transforms, physical thresholds and source artifacts. EmbodiChain owns the sole collection World, rolling scheduling, measured evidence, native sessions, confirmed receipts and export.

  • run_task_program_expansion binds closed configuration, bounded scene/grasp declarations and the original measured criteria to the existing native workload engine. Native preview launch/wait/stop APIs own the service lifecycle; integrators retain only their workflow entry and task data. Pure loaders avoid simulator imports.
  • GymExpansionWorkload and run_gym_expansion_workloads compose WorkloadProgramExecutor with the existing native host, episode assembler and artifact sink. Capacity is W × B; groups retain independent targets, candidate/time budgets and rolling row refill. One bounded writer thread exports frozen groups without World access.
  • Capture the seeded nominal scene in the same World. Freeze reset scene events before rolling admission and reject resets that advance physical time outside the owner loop. Explicit workload-local grasp contact frames retain strict TCP, mesh, drive and command binding checks without mutating the live robot.
  • Native phase paths, same-grid plan transforms and pose_nms_scope replace downstream global motion/NMS overrides. Task assets, calibration values, perturbation ranges and acceptance thresholds remain task-owned.
  • MeasuredExpansionValidator retains independent geometry checks or full native physics substeps, including sustained finger contact and bounded drift. Missing contact evidence and unknown actors reject acceptance.
  • Native result/distribution previews use recorded full models, rigid Mesh/USD visuals and physics-free solid/ghost playback. Confirmed per-episode LeRobot exports retain T+1 states and portable hashed source/plan evidence. Browser transport is loopback HTTP JPEG; WebRTC remains TODO.

Task-pose expansion and dataset distribution analysis

Tasks can independently select bounded pose proposals or workspace-conditioned joint sampling, and FIFO or receipt-confirmed task-pose feedback. A default-physics UR5 pick-place deployment exercises open and obstacle layouts. Preserve existing physical validation, per-row budgets and owned observations/targets; fix scene-relative Place target grounding and reject nonfinite transport evidence.

The offline distribution report reads confirmed expansion artifacts and provides world-XY layout maps, start/end joint distributions, full SO(3) orientation features, measured object/joint motion and recorded-URDF TCP FK with explicit gripper calibration. A styled HTML entry links filters to trajectories and before/after comparisons. JSON/JSONL and 100-record detail pages provide the same evidence to Agents; the browser caches at most two detail pages and paginates sample lists, matrices and orientation tables. Share the complete report directory, not just the HTML entry.

Validation and limits:

  • Existing formal pick-place matrix: 40 jobs, 5,120 executions, 4,591 accepted-and-confirmed physical episodes. Open: 2,408/2,408 orientation/TCP records; obstacle: 2,183/2,183. Before snapshots use variant A (585 / 512 episodes); after snapshots use the cumulative A+B+C+D union. This is a dataset-composition comparison, not a learning-performance claim.
  • All page records match JSONL; aggregate orientation/joint counts, TCP availability, displayed TCP values, page boundaries, filters and mobile layout pass consistency checks on both full scenes. Six focused numerical/ownership/pagination tests pass.
  • Formal sampling acceptance remains open PASS / obstacle FAIL. Median paired workspace-feedback coverage gain is +39.60% (open) and +32.63% (obstacle); obstacle acceptance-rate loss is 5.47 percentage points, exceeding the unchanged 5-point gate. Do not interpret improved coverage as universal acceptance improvement. The formal matrix precedes the previously documented compatibility/invalid-input hardening; final B=4 smoke and focused tests cover the hardened code.
  • Orientation bins are 30-degree Cartesian rotation-vector bins, not equal-volume or symmetry-aware. TCP is FK-derived, not directly sensed. Unsupported calibration is explicit. Lightweight indices and extracted descriptors remain O(N); detail trajectories are loaded by page. No full-suite, new throughput or policy-quality claim.
  • Black 26.10.1 (black .), API coverage 5014/5014, context validation and whitespace checks pass. Expansion/Task Program/distribution regression: 223 passed, 15 deselected (native simulation/GPU-marked cases excluded from this rerun; existing full-data and physical validation remains separately scoped). Affected expansion and scene context was revised; no external dependency was added.

Configured/cohort APIs remain compatible; workload collection uses an explicit public facade. No private task assets or downstream workflow orchestration are included. Dependency: greenlet>=3.2,<4; physics backend/device remain deployment-owned.

The current follow-up preserves reset hold targets during real settling, retains configured two-hand grasp commands, and batches explicitly supported preceding-seed SSIK paths on the World owner. Optional bounded idle-row reuse gives queued work priority and permits one additional competitor per failed attempt; W × B and the original attempts/time/step/settling/byte limits remain fixed. Both low-level and Task Program defaults retain fixed lanes because measured CPU workloads did not establish a throughput benefit. Reuse requires explicit engine.reuse_idle_rows: true.

Only confirmed native receipts earn quota. Competing peers stop through the canonical acknowledgement handshake and retain their real program outcome and T+1 partial evidence. Pending writes cannot be cancelled or double-reserved. For a terminal runtime with an unconsumed final motion, the joint controller owner can supply its actual applied position targets as an explicit safe hold; pending motion itself is never relabelled safe. Without an owner-proven hold, the existing strict failure remains.

Conditional scene layout expansion

Generate support-tree layouts from authored anchors or workspace samples. The
scene-first and robot-first sources share the same full-state IK/FK, path
collision and measured acceptance checks. The bidirectional source alternates
both directions within one proposal budget. Bounded pose sampling remains the
default. Candidates and accepted artifacts retain independent ownership.

Validation

DexEChain spatial integration and installed qualification (2026-11-18)

Pinned integrated SDK: c67c7218833a7e491c6e04be9628a9a9755776bb, actual DexSim 0.6.0rc2. The downstream extension retains a 30-line expansion entry and forwards offline reports to public report_expansion_distribution; the six obsolete engine/preview modules remain absent. Five explicit experimental profiles use one collector process, one World, W=1/B=8. Original regular profiles remain unchanged.

Generic follow-ups keep a task-authored nominal reset seed independent of proposal/runtime streams, support named measured entity/segment coverage, preserve admission-only wall deadlines for already active trajectories while retaining shared call limits/cancellation/max_steps, and package the offline HTML template. Installation identity grouping absorbs only measured base readout jitter at 1e-6 component resolution; declared robot/calibration configuration remains exact and raw states, FK and physical acceptance are never rounded. Concurrent task-identity, layout-admission, simulator-free declarations and event-history changes are preserved by merge.

Fixed nominal scene: four proposal seeds, FIFO versus receipt feedback

The task's authored nominal seed is fixed; proposal seeds 2, 7, 18 and 29 rotate policy order. Every independent cohort targets eight receipts. Original scene perturbation bounds, per-trajectory control steps, settling and all contact/physical/motion/terminal criteria remain unchanged. Attempt allocation scales the original per-operator limit by the eight-target quota. Coverage uses the union of confirmed initial cells across the four seeds; fixed terminal tracking noise earns no novelty.

Task FIFO / feedback confirmed (each target 32) FIFO / feedback episodes/min Initial confirmed cells Initial cells/min change
CobotMagic stack 32 / 32 9.123 / 9.292 40 → 36 -8.3%
Tianji handover 27 / 27 0.679 / 0.619 14 → 15 -2.2%
Piper book 32 / 32 3.527 / 3.354 13 → 16 +17.0%
Bottle relay 32 / 32 2.134 / 2.086 15 → 15 -2.2%
Bimanual marker 32 / 32 0.873 / 0.953 30 → 34 +23.8%

Tianji feedback does not pass full-quota qualification. The latest integrated c67 wheel independently collects 8/8 for stack, book, bottle and marker, and 7/8 for handover (39/40 total). These incomplete data are not promoted to success or pooled with reruns. Feedback improves initial cells/min here for Piper (+17.0%) and marker (+23.8%); the other tasks have no net richness-rate gain. Existing defaults remain unchanged.

Version scopes are explicit: the comparison's 32 non-handover cases were measured on 657daec6c and all finished before the admission cutoff; the active-budget view is equivalent on those paths. The eight affected handover cases were remeasured on f38aacba9. All five tasks have a separate latest-c67 eight-row collection; the branch's subsequent merge is covered by installed integration and focused checks. Do not call the earlier matrix a fresh four-seed measurement of every merged-head path.

  • Latest installed runtime/GenSim/full-layout checks: 7 passed in 427.63s. Both complete GenSim workflows pass build → prepare → replay → evaluate, with 369-step experts and dynamic replay. Three conditional layout strategies each exercise one World/eight actual rows and confirmed T+1 readback/report generation. Workspace-conditioned sampling separately passes a native UR5 eight-row 8/8 smoke; it is not a five-private-task workspace qualification.
  • Merged focused checks: 91 passed; downstream configured-semantic/entity/phase checks: 13 passed. The admission/count/cancellation regression retains real cancellation and count limits. No full-suite or GPU-physics claim.
  • Every accepted private-task parent passes independent receipt/hash, original physical predicates, exact World control span, T+1, finite values and real LeRobot readback. Piper retains all 71 sustained two-finger samples, full substeps and 0.012m / 0.12rad drift bounds. Named coverage boundaries are independently recomputed.
  • Actual VCS metadata and reviewed bytes match: 1,164 upstream Python files, 103 downstream Python/JSON/YAML files and both HTML templates; expansion entry remains 30 lines. Packages were built from the published local Git checkout, not misrepresented as HTTPS installation.
  • Five current full-model/ghost previews use recorded state, zero playback physics steps and unchanged source JSON hashes. The MP4s compress progress into eight seconds and preserve their exact frame/time source audit. Offline HTML/JSONL bundles remain the primary detailed distribution view.
  • Black 26.10.1, API 5053/5053, context and Sphinx dummy pass. Pre-existing environment/documentation warnings are retained separately.

Preserved prerequisites, negative outcomes and timing limits

Fresh fixed 2×2×2 prerequisites (W=8/B=1, SDK 9e29ad63a → 0557cdcdf, two runs/task) produce 75/80 → 76/80, 3885.02 → 3124.98 collection seconds and 1.158 → 1.459 episodes/min. This is a version-scoped compatibility comparison, not a current-head two-round throughput claim. The old marker second run remains 7/8 at 1272.25s; the two fixed authored stack cells remain incomplete.

The original coupled-seed forty-policy-case experiment is retained in full: changing seed moved the stack's nominal layout by roughly 8–13cm despite only ±1cm declared local jitter, and three seeds yielded 0/8 on each policy. The earlier fixed-nominal experiment that hard-stopped admitted handover rows at the admission deadline is likewise retained; it motivated the admission-view correction. Neither failure set is replaced by selected good seeds. Asset-startup, excluded configuration/resource pilots and the installed missing-template failure remain separate diagnostics.

Collection timing includes World setup, proposal/screen/planning work, physical execution, original acceptance, export and SDK readback. Experiment queues, input assembly, independent audits and offline reports/previews are outside that clock. CPU physics, matched logical affinity within each pair and one numerical thread; host, SMT, bandwidth and GPU remain shared. Four-seed observed bins do not establish feasible volume, statistical significance, a learning benefit or universally optimal throughput. Root layout proposals still require explicit XY-only domains/task phases; fixed workspace grasp priors do not support arbitrary dynamic affordance replacement.

Detailed versioned qualification and private evidence references: DexEChain record.

Conditional scene layout expansion (2026-11-18)

Commit 910db9463 adds the three-task comparison on Default CPU physics with
B=1 and sensors disabled. Four seeds rotate the strategy order. Each task and
strategy executed and committed 24/24 episodes with the same physical
thresholds. All 288 receipt readbacks passed the T+1 audit, and all 108
workspace anchors passed their layout verification.

Task Bounded Scene first Robot first Bidirectional
Pick and place 29 / 34.41 s 29 / 33.51 s 28 / 34.19 s 28 / 33.68 s
Stacking 24 / 39.51 s 29 / 39.21 s 24 / 39.19 s 26 / 39.45 s
Dual-arm RGB sorting 27 / 88.83 s 24 / 87.39 s 25 / 86.97 s 23 / 86.16 s

Each cell shows covered marginal XY bins out of 32 and median wall time per
six-episode run. Coverage differs by task. Timing differences are small in this
comparison. The final PR head passed 67 focused tests, Black 26.10.1, the API
documentation gate and project context validation.
Configuration and reproduction guide.

Installed comparison and release policy (2026-11-17)

Matrix baseline: d083f616fef06efde1f9441f091ac457eabd0fd8; matrix after: db388b3b7732442dacd58a2c63c37ced1e021eea, explicitly enabling idle-row reuse. Release pin: 9e29ad63abb96af1026b82ad72353f52eac38d0f; its only subsequent behavior change restores the Task Program default to fixed lanes. The matrix remains evidence for explicit reuse, not a five-task measurement of the latest default. Both matrix sides and the release use actual VCS-built packages and DexSim 0.6.0rc2, matching the SDKs' declared dependency. The downstream entry now rejects the previously accepted rc1 binary. Packages were built from the published local Git checkout after HTTPS clone failures; actual commit metadata and reviewed source bytes are verified.

Each run owns one collector process, one World and W=8/B=1 physical capacity. Two bounded position proposals, authored/non-default grasp groups and nominal/via-point modes retain independent target-one quotas. Base seeds, sampling bounds, protected phases, budgets and all original physical/contact/FK/terminal rules are unchanged. Logical retry seeds advance by workload ordinal on both SDKs. Task sampling identity excludes the SDK/scheduling identity. Actual scene proposals compare exactly across all four runs per task. Failed cells remain explicit.

The existing two d083 baseline rounds are retained; the current after rounds are new measurements, not a newly executed ABBA experiment. Matched CPUs: 128–143 for marker and 160–175 for the other tasks; visible GPU 3, one numerical thread, tmpfs. Two experiment groups may co-run on disjoint CPUs on the shared host. Physics, state and task solvers are CPU; this is not a GPU-physics qualification. Collection wall time includes World initialization through complete export and readback, excluding repeated import/input assembly, experiment queues, reused builds and independent audits. All speculative/cancelled work remains inside that measured collection cost.

Task Before two runs: seconds / qualified After two runs: seconds / qualified Effective output change
CobotMagic stack 56.68s / 6/8; 57.15s / 6/8 56.54s / 6/8; 58.00s / 6/8 -0.6%
Tianji handover 280.22s / 8/8; 270.30s / 8/8 286.05s / 8/8; 309.36s / 8/8 -7.5%
Piper book 121.51s / 8/8; 119.14s / 8/8 120.50s / 8/8; 123.27s / 8/8 -1.3%
Bottle relay 200.42s / 8/8; 161.52s / 8/8 239.01s / 8/8; 171.55s / 8/8 -11.8%
Bimanual marker 456.49s / 8/8; 476.08s / 8/8 501.08s / 8/8; 658.55s / 8/8 -19.6%

Aggregate qualified output: 76/80 → 76/80; summed collection wall time 2199.51 → 2523.93s; effective output 2.073 → 1.807 episodes/min (-12.9%). These two fixed-seed rounds describe this configuration; they do not prove universal speedup or bitwise physical determinism. Administrative cancellations are reported separately and earn no receipt. Incomplete quotas are not promoted to success or pooled across runs.

  • Every accepted trajectory independently passes native receipt/hash, original physical predicates, T+1, exact World control span and real LeRobot SDK readback. Cancelled evidence additionally checks raw hashes, actual spans/T+1, canonical outcome/phase ranges and no persisted receipt. Piper retains all 71 consecutive two-finger samples, required physical substeps and 0.012m / 0.12rad drift bounds.
  • Native lifecycle/configuration/real collector regression: 41 passed. Focused abort/safe-hold/real reuse checks: 12 passed. The known-bad scheduler copy is rejected by the unexecuted-hold regression. The restored fixed default passes the real declarative collector check (42.54s). Installed release runtime/reuse and both complete GenSim workflows: 5 passed in 251.09s; both workflows include 369-step experts and dynamic replay. No full-suite claim.
  • API coverage 5013/5013, context and Black 26.10.1 pass. Downstream expansion remains a 30-line entry; six obsolete engine modules remain absent. Task assets/calibration stay downstream.

Preserved negative intermediate and diagnostic evidence

The complete supported-runtime cba37a4ae fixed-lane experiment produced 76/80 on each side but 2199.51 → 2616.71s and 2.073 → 1.743 episodes/min (-15.9%). Bimanual after rounds were 437.29/809.37s; this negative result is retained. It motivated idle-row reuse after active-trajectory utilization fell from about 64.6% to 50.2%. The initial idle-row revision's bottle cancellation failure is retained and not counted as successful qualification.

Earlier rc1 frozen-candidate runs do not satisfy the SDKs' declared exact rc2 dependency and stay historical diagnostics. Frozen retries repeatedly reused the same expanded candidate; the current comparison applies ordinal-based proposal advancement to both sides. Increasing captured IK search from 128 to 2048, high-precision frame conversion and enhanced physics determinism did not establish a cure. Scalar/native path queries match exactly on the captured inputs; isolated CPU B=1 kernels improve while one B=8 path does not, so no general kernel-optimality claim is made. A same-World contact capture confirms actual wrist-pitch-link/table rejection, not a relaxed permission.

Detailed private-task data and versioned evidence: downstream qualification record.

Historical revision-specific qualification records

Eight-environment collection fixes and performance (d083f616f, 2026-11-17)

HandOver now carries context.env_ids through all source/destination FK calls and diagnostic IK. The original eight-row run failed every semantic-call preparation with Expected 8 but got 1 and produced no effective actions; the same fixed combinations now execute under the original physical checks.

Measured acceptance reuses existing batched Newton queries and measured-predicate scopes. Compatible geometry shares a checker, selected rows are compacted within the existing contact budget and reports return to their original rows. Native contacts still sample every physics substep; ordered events, consecutive counts, failures and hold evidence remain row-owned. Shape snapshots are acquired only for support-bound predicates, preserving native-contact admission without a new homogeneous-topology requirement.

Unmodified built-in joint encoders write a complete multi-destination frame into one owned output; subclasses and overridden methods retain public dispatch/registration order. JSON publication keeps identical sorted finite text, fsync and atomic replacement while using the C encoder. Compatible segment metadata fields share immutable nodes; mutable public projections remain detached.

The final installed HTTPS VCS pair is pinned to d083f616fef06efde1f9441f091ac457eabd0fd8; source fixes are in bf4a25422. Main 9474d998d and concurrent test-only 33fe553ae are retained without rewriting published history. The downstream entry is still 30 lines, and its six obsolete engine modules remain absent.

Both sides rerun the same 2 position proposals × authored/non-default grasp × nominal/via-point trajectory recipes, one target per exact combination, W=8/B=1 in one collector process/World. Seeds, axis selections, candidate/time/control-step budgets, physical parameters, protected samples, full substeps and all acceptance thresholds are unchanged. Fixed failed cells are retained instead of replaced.

Task Before seconds / qualified After seconds / qualified Effective throughput change
CobotMagic stack 110.62 / 6/8 61.25 / 6/8 +80.6%
Tianji handover 15.71 / 0/8 251.43 / 8/8 0→8 restored; ratio not applicable
Piper book 129.11 / 8/8 122.38 / 8/8 +5.5%
Bottle relay 213.90 / 4/8 169.04 / 4/8 +26.5%
Bimanual marker 830.37 / 8/8 1252.66 / 7/8 -42.0%

The primary matrix is 26/40 → 33/40, with 1299.72 → 1856.76 summed collector seconds. Effective aggregate output is 1.200 → 1.066 qualified trajectories/min (-11.2%): no net aggregate throughput gain in this run. Bimanual marker misses one fixed cell after 20 attempts and exhaustion of the original 1200-second admission budget; its held-object/placement failures remain rejected. Any independent recheck is recorded separately and never replaces this table.

A separate complete bimanual rerun with byte-identical specification, SDK, N=8 and CPU/GPU allocation passes 8/8 in 385.48 collector seconds with 10 attempts. Its independent audit passes. The original failed cell succeeds on its first attempt in this rerun. This demonstrates substantial run-to-run planning/physics variation; the rerun does not replace the primary 7/8, timing or aggregate throughput above.

One cold-process measurement per setting on the same shared RTX PRO 5000 host, visible GPU 1, CPU 96–111 and one numerical thread. Physics/state/solvers are CPU; both sides use tmpfs. Timing includes World initialization, collection, original measured acceptance, export and readback, excluding reused builds, experiment assembly and independent audits. Failed handover time is failure cost, not completed-work throughput. Counts and retry costs remain explicit; aggregate changes include restored handover functionality and cannot be treated as isolated scheduler/kernel speedup or statistical stability.

  • Every accepted episode passes independent native receipt/byte-hash, exact combination, T+1, control-span, protected trajectory, motion/FK, measured terminal and real LeRobot readback audits. Piper retains 71 two-finger samples and 0.012 m / 0.12 rad drift limits.
  • 16 encoder, 40 data/journal/media/numerical collision and 2 real port checks pass. The selected-row HandOver check is a short execution regression, not full task qualification.
  • Installed downstream runtime/SDK checks: 4 passed. Both complete GenSim workflows pass build/prepare/replay/evaluate and dynamically replay all 369 steps.
  • Three independent review-pr passes complete; public-method dispatch and needless shape-export dependency findings were fixed. API 5013/5013, Black and context checks pass.
  • Newton heterogeneous native-contact task qualification, a full test suite, GPU-physics performance and independent GPU memory peaks were not measured. Rejected early setup/compatibility commands and unsupported Default-backend variant tests remain preserved outside passing results.

Detailed downstream record: fixed-grid comparison and acceptance. Earlier version-scoped evidence below is retained.

Verified GenSim release and complete workflows (565d98129, 2026-11-16)

The original release failure was physical: the second finger contacted the support and stayed around 13 mm from open despite a zero target for two seconds (the existing threshold is 4 mm). An isolated post-failure diagnostic lowering the support by 50 mm lets the same hand open to about 13 µm. Formal qualification keeps the original scene and physics.

GenSim now defaults to 0.02 m release clearance above the measured resting target, followed by a 0.4 s fully-open command hold before the unchanged release gate. PlaceOptions.release_settle_duration is keyword-only, defaults to zero outside generated GenSim behavior, rounds upward on the actual control cadence and combines with explicit steps using the longer hold. Explicit zero clearance/duration and explicit step declarations retain their ownership; external nominal sources keep their original timing. The zero default preserves sealed registration fingerprints.

Neither physical acceptance thresholds nor monitors were relaxed. Independent final-artifact comparisons confirm the original program trees, post-policies, final resting targets, effect-monitor declarations and 0.004 m grip evidence settings are unchanged. Recorded valid constraint=False measurements release the gate; final stability and target validators pass.

A fresh package from the published GitHub commit passes both full installed workflows:

Pipeline build prepare replay evaluate
EmbodiChain prepared scene pass pass pass pass
DexGen materialized handoff pass pass pass pass

Each expert completes 369 control steps; evaluate dynamically replays all 369. Both replay/evaluate videos contain 370 frames at 960×540 / 25 FPS. The two semantic graph hashes, task semantic hashes and object bindings match; every binding retains its original plan digest. Trajectory/terminal states, measured gates, post-stability, final targets and registered artifact hashes pass the independent audit. This remains a scripted Task Engine / fixed-handoff integration, not live photograph-generation or model-quality qualification.

  • 4 focused native planning/configuration cases and 2 registration-fingerprint cases pass.
  • 50 unique installed configuration/workflow/SDK-readback scenarios pass, plus the 2 complete physical workflows. A host disk-exhaustion readback error was rerun after cleanup; initial logs remain.
  • API coverage 5010/5010, context and Black pass; Sphinx dummy succeeds with 889 documentation/dependency warnings.
  • DexEChain only updates the SDK pin, full-workflow tests and documentation. Its 30-line expansion entry and upstream ownership remain intact.
  • Current DexSim still asserts during long asset-path initialization: the 261-byte OBJ case fails before execution, while short output roots complete. Final qualification uses a short temporary root; this limit is recorded separately from the repaired release failure.

Current downstream validation: release diagnosis and complete-workflow record. Older failure, five-task and throughput evidence below remains scoped to its original revisions.

Combined regression pinned to 8c244cdc9 (2026-11-16)

The placement fix expands each original phase mode with all of its translation keyframes. A real UR5 planning regression fails on f3e08ae6 and passes with the fix in both joint→Cartesian and Cartesian→joint directions. Main 8170fca18 is merged; its sampled Cartesian approach stays available and rejects simultaneous explicit phase paths.

DexEChain MR !1412 now fixes this tested SDK revision, removes the obsolete desktop-viewer timeout protocol and its three synthetic-state cases, and retains the single 30-line expansion entry. GenSim builders use verified TaskSpec interpretation steps and pass the original plan digest into upstream binding rather than duplicating plan validation.

The first five-task matrix retains all original physical criteria, perturbation ranges, quotas, seeds, attempt/time budgets and W × B capacities:

Task Environments First run qualified
CobotMagic stack 5 10/10
Tianji handover 1 10/10
Piper book 4 10/10
Bottle relay 5 9/10
Bimanual marker 2 10/10

The first matrix is 49/50. Bottle affordance exhausted eight attempts with one confirmed episode; the CLI correctly failed its quota. A separately recorded complete bottle run, with the same parameters/seed/capacity/budgets, passed 10/10. Each task therefore has a complete successful run; the selected 50-episode set uses one whole run per task, with bottle taken from that recheck. Receipts or operators are never pooled between runs. The recheck used visible GPU 2 and CPU 32–47, separately from the serial primary matrix on GPU 3 / CPU 0–15. Physics remained CPU.

Independent audits cover receipts and byte hashes, real LeRobot readback, T+1, PID/World identity, W × B, control boundaries, protected samples, scene/motion/FK limits and measured terminal/contact criteria. Piper retains 71 two-finger samples, every required physics substep and 0.012 m / 0.12 rad drift limits. The Cobot browser reuses these saved results: filtering 10→2→0→2, timeline/play/stop, zero physics steps and unchanged hashes for all 569 run files.

Both packaged GenSim pipelines ran full workflows: build/prepare succeeded, pickup completed at steps 0–180, and measured release verification rejected place at 180–283. Both preserve the historical segment_validation_failed, return nonzero, skip evaluate and save no successful expert trajectory. Their graph/semantic hashes and object bindings match, each binding matches its own original plan digest, and both videos are 960×540 / 25 FPS / 284 frames. This uses the scripted Task Engine and materialized DexGen handoff, not a live image-generation/model-quality qualification.

  • Final installed downstream pair: 52 passed, one legacy bridge deselected.
  • Merged native source: 22 focused checks + 3 real integration checks passed.
  • Black, context and API coverage 5010/5010 pass; Sphinx dummy succeeds with 894 documentation/dependency warnings.
  • 15 Markdown files, 83 local links, 51 shell examples and 42 documented CLI invocations checked.
  • Source Python and task configurations match the qualified wheel. No new throughput, GPU-physics, full-suite or repeated-run determinism claim is made.

The PR also contains concurrent follow-up 2dfcbed03 for hold-clock accumulation and configuration-owned joint selections. Its own commit records separate validation; it is not part of this downstream pin or five-task qualification.

Detailed downstream evidence: combined regression record.

Historical entry extraction and qualification

Entry extraction follow-up (7f9706409): downstream expansion shrinks from seven modules / 1,071 lines to a single 30-line workflow entry, with six obsolete modules absent from its built wheel. A fresh VCS-built installed pair passes 55 configuration/workflow/GenSim/real LeRobot checks and the complete collect→export→native browser→filter→stop read-only integration. Source checks also pass real five-operator collection and native pure-loader / real two-environment declarative collection. Original numerical/physical thresholds remain unchanged; this is not a new five-task throughput measurement. API coverage is 4,960/4,960 and Sphinx dummy succeeds (886 dependency/documentation warnings; no new entry-API warnings).

Historical full five-task qualification wheel: 78dc36c41fa804d9de17761522499eecd2c8ad0b. Five real downstream tasks each completed 10/10, 50/50 total, retaining original physical criteria, per-task attempt/time budgets and equal environment capacity. Independent native receipt/content-hash and real LeRobot SDK audits checked every accepted episode, T+1, continuous control boundaries, protected phase samples, motion/FK limits, scene bounds and measured terminal/contact criteria. Piper retains all 71 required two-finger samples and the 0.012 m / 0.12 rad drift limits.

One fixed-seed cold collection-stage comparison on the same machine, visible GPU 3, CPU affinity 0–15 and one numerical thread uses the original CPU physics deployments. Timing runs from expand-stage start to durable worker result, including preparation/World startup, collection, acceptance, LeRobot export and readback; shared build and outer artifact registration are excluded consistently.

Task Environments before/after Before seconds / qualified After seconds / qualified Effective throughput change
CobotMagic stack 5/5 114.89 / 10/10 143.27 / 10/10 -19.8%
Tianji handover 1/1 296.31 / 10/10 199.07 / 10/10 +48.8%
Piper book 4/4 151.82 / 10/10 203.60 / 10/10 -25.4%
Bottle relay 5/5 235.60 / 9/10 259.13 / 10/10 +1.0%
Bimanual marker 2/2 1065.44 / 10/10 1107.92 / 10/10 -3.8%

Aggregate effective throughput is 1.577 → 1.568 qualified episodes/min (−0.6%): this migration establishes native ownership and compatibility, not a general throughput improvement. The original bottle run missed one affordance episode; incomplete runs and all retries remain explicit. Candidate counts and physical success rates vary across runs, so the aggregate difference is not an isolated scheduler speedup or a GPU-physics qualification. Memory peaks were not sampled.

The measurement controller received SIGTERM before the last outer CLI record finalized. Its independent collection worker completed and persisted all 10 episodes. The original manifest and recovery evidence are retained; artifact registration was restored from the successful worker result, without fabricating a CLI exit code or timing. All ten measurements use the same durable-worker boundary above. A redundant qualification recheck was cancelled after the primary final-wheel matrix met every quota; its partial evidence is separate.

  • Real native four-row execution and two-row facade coverage checks rolling refill, queued groups, safe rejection/hold, confirmed persistence, T+1 and LeRobot export. The facade case also exercises different contact sampling conventions, unchanged actual robot bindings, default strict installation rejection and incompatible drive rejection before another World is built.
  • Extra real W=2/B=2 downstream collection passed 10/10 and the independent physical/data audit. Two final-wheel Chromium viewers passed filtering 10→2→0→2, timeline/play/pause, zero playback physics steps and real full-model rendering.
  • 63 focused checks passed across native execution/journal, downstream runtime/end-to-end, API documentation and context tooling. Earlier operator/phase/preview/NMS regression evidence remains applicable; no full-suite or repeated-run determinism claim is made.
  • Black 26.10.1, whitespace and context gates pass. The preceding revision had API coverage 4,946/4,946 and a successful Sphinx dummy build; current entry APIs and documentation validation are listed above. The two wheel builds match the reviewed Python sources; legacy dexechain is not imported.

Type of change

  • Enhancement and correctness fixes

Checklist

  • Repository-pinned Black and API/context gates
  • Focused behavioral and real workflow checks
  • Original physical acceptance and confirmed receipt accounting
  • Incomplete quotas, negative results and measurement limits retained

@yuecideng yuecideng added enhancement Improvements to existing capabilities; use feature for new capabilities. physics Physics backends, dynamics, contacts, fluids, deformables and physics solvers. kinematics FK, IK, Jacobians, reachability and kinematic solvers; not physics solvers. motion gen Robot motion planning, MotionGenerator, trajectory generation, retiming and expansion. dependencies Dependency versions, upgrades and compatibility. sensor Sensor models, attachment, lifecycle, extension interfaces and observation contracts. data-pipeline Data collection, ODS, sampling, recording, persistence, conversion and analysis. task-program Task Program language, semantics, compiler, runtime and integration contracts. labels Nov 14, 2026
@yuecideng
yuecideng marked this pull request as ready for review November 14, 2026 09:51
Order permission queries after the latest sensor sample on the caller's CUDA stream without resampling or advancing histories. Cover pending forbidden contacts and dropped evidence in standalone and history sampling paths.

Validation: 75 targeted tests passed; Black 26.10.1, public API documentation coverage and agent context checks passed.
@yuecideng yuecideng changed the title Add single-World workload execution and batch checks Standardize expansion host, acceptance and ghost comparison Nov 14, 2026
Share Default CPU numeric observations within bounded command-gather scopes, reconstruct immutable standard targets, and transfer fresh internal command ownership while preserving public snapshots and validation.

Keep native multi-chunk grasp geometry and reductions resident on CUDA within each query, preserving samples, thresholds, stream ordering and independent results. Add focused regression coverage and a reproducible runtime comparison driver.

Validation: 201 focused tests including CUDA and 42 context tests passed; Black 26.10.1, API docs 4924/4924 and diff checks passed. Sources match the final benchmark manifest.

Forward/reverse sensor-free UR5 execution versus main 0119171 (seconds): small main 10.34-10.45, off 9.07-9.31, on 8.31-9.08; medium main 49.24-57.12, off 47.19-50.90, on 48.66-49.57; large main 251.77-260.14, off 189.57-196.42, on 192.07-200.29. All 492 program attempts accepted. Default CPU physics retains legacy CUDA grasp classification; these shared-host single-seed results are not physical qualification.
Restore the existing Default CUDA backend policy. Remove the newly introduced public selector and its ordinary-reporting test variant, retaining CPU contact permission coverage and documenting Direct GPU static-identity limits.

Validation: 53 configuration/contact tests including CUDA and 42 context tests passed. Black 26.10.1, API docs and diff checks passed.
@yuecideng yuecideng changed the title Standardize expansion host, acceptance and ghost comparison Standardize native expansion collection, acceptance and previews Nov 15, 2026
Preserve legacy catalog identities when phase paths are inactive, while keeping explicit phase declarations in registration fingerprints. Align preview error coverage with aggregate input support and update state test doubles for Direct GPU queries, including publication guard coverage.

Validation: reproduced all 11 failed CI cases; 65 focused tests passed; broader Task Program and command/context suite 327 passed, 3 GPU cases skipped. Black 26.10.1, API docs 4963/4963, context map and diff checks passed. Existing context already describes inactive-default fingerprint compatibility; no context update required. Main synchronized through 438e111.
Move the existing pre-Scene collection after clearing facade-owned data and entity references. Cyclic wrappers newly orphaned by those clears must finalize while their Scene and World parents are still alive, instead of being collected during a later teardown.

Extend the existing deferred-destruction test with a cyclic view finalizer; both registry-ownership cases fail before the change. Keep backend synchronization, ownership checks, physical validation and GC count unchanged.

Validation: 288 manager CPU tests passed; the full 149-node failed CI GPU shard passed 143 with 6 pre-existing capability skips on pinned DexSim 0.6.0rc2. Black 26.10.1, API docs 4963/4963, context and diff checks passed. Lifecycle context already requires child-before-parent cleanup and remains accurate. The reported native abort itself was not deterministic locally; this fixes the demonstrated lifetime gap at its cleanup boundary.
A finished runtime may still have its last ordinary motion awaiting env.step. Keep the default fail-closed abort contract, but allow the joint controller owner to supply actual applied target positions when no runtime safe hold exists. Never reuse the pending motion or acknowledge before real physics consumption.

Retain the interrupted phase when terminal fields are empty, and keep the last actual scalar controller metadata and row identity in cancelled diagnostics without per-step serialization. Validation: 12 focused abort, safe-hold and actual reuse cases pass on rc2; full bottle trigger and repeated five-task qualification follow on the installed revision. Independent reviews completed.
Resolve scene preparation with the same precedence as EmbodiedEnv: explicit config first, registered defaults second. Normalize registered declarations through an owned FunctorCfg before shared-World compatibility checks.

Extend real deployment coverage for mismatched and identical registration defaults, explicit overrides, and equivalent explicit/registered declarations. Preserve single-workload validation behavior and document the effective-hook contract.
Generate layouts from authored anchors or workspace samples and alternate proposal directions within the existing budget. Retain the default bounded sampler, full task screening and independent measured acceptance. Compare all four strategies on pick/place, stacking and dual-arm RGB sorting.

@han-xudong han-xudong left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seven P2 findings from the follow-up review, covering comparison identity, configuration loading, layout proposals, budget exhaustion, architecture generation and event validation.

@han-xudong han-xudong left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed this head and the changes since the previous review. All seven previously reported findings are addressed, and no additional actionable defects were identified in the reviewed changes.

Coverage included rolling layouts and follower constraints, budget exhaustion, measured phase coverage and receipt accounting, nominal/proposal seed separation, simulator-free configuration schemas, report identity and packaging, and the architecture evidence update. Independent robotics, agentic/engine and API reviews completed. Focused CPU tests and isolated production-code probes passed.

Validation limits: local native integration tests could not run with the installed DexSim version. At submission, CI lint, Documentation tests, offline tools and LeRobot export had passed, while build/test and the LeRobot SDK matrix were still queued.

@yuecideng
yuecideng merged commit b715f35 into main Nov 19, 2026
2 of 8 checks passed
@yuecideng
yuecideng deleted the codex/expansion-workloads branch November 19, 2026 09:46
@yuecideng

Copy link
Copy Markdown
Contributor Author

The requested skill-boundary diagnosis and default controlled-stop follow-up is now in draft PR #1250 (commit 532fe6e), based on main because #1192 has merged and its source branch was deleted.

The PR includes paired raw six-joint position/velocity curves and before/after/after/before Default-backend pick-place results. Residual control-grid measured speed improves, but reference jumps remain and the new tolerance rejects an additional workspace-sampled episode per repeat. Existing validation and grasp/release gates are retained; no criterion was relaxed to recover success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

benchmark Benchmark protocols, suites, metrics, evaluation tools and technical reports. data-pipeline Data collection, ODS, sampling, recording, persistence, conversion and analysis. dependencies Dependency versions, upgrades and compatibility. enhancement Improvements to existing capabilities; use feature for new capabilities. kinematics FK, IK, Jacobians, reachability and kinematic solvers; not physics solvers. motion gen Robot motion planning, MotionGenerator, trajectory generation, retiming and expansion. performance Performance, memory, throughput and synchronization-cost improvements. physics Physics backends, dynamics, contacts, fluids, deformables and physics solvers. sensor Sensor models, attachment, lifecycle, extension interfaces and observation contracts. task-program Task Program language, semantics, compiler, runtime and integration contracts.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants