Skip to content

Compile fitted Pickups into native Task Programs - #1240

Draft
han-xudong wants to merge 56 commits into
mainfrom
feat/real2sim-fitted-task-program
Draft

han-xudong wants to merge 56 commits into
mainfrom
feat/real2sim-fitted-task-program

Conversation

@han-xudong

Copy link
Copy Markdown
Collaborator

Description

Compile an accepted scene-bound Pickup fit into a sealed, standard Task Program. The export retains the qualified deployment, control clock, and attributed source intervals. It binds each fitted Pickup to its own named policy and preserves remaining source spans as explicit DemoSkills with measured entry and terminal contracts.

  • Read portable v1 and v2 bundles while keeping measured state, submitted commands, and the qualified reference distinct.
  • Bind object, robot resource, source span, and compiled invocation identities. Retain the source event-to-control-tick map and reject modified deployment files.
  • Pass invocation policies, nominal sources, and validators through native collection. Allow a plan transform in an authorized free phase and reject a conflicting override on the same invocation.
  • Keep explicit observations and held-object evidence in the handoff to the monitored DemoSkill. Evaluate the rollout before any downstream dataset can consume it.

Stack

Type of change

  • Enhancement

Validation

  • Black 26.10.1 and the public API coverage check passed.
  • 186 focused native tests passed across fitting, DemoSkill, Task Program, expansion, and the consuming integration. Another 54 documentation and agent-context checks passed.
  • The added cases cover source-span reconstruction and attribution, including original command/measurement separation, plus the compiler-to-binding path. Existing tests cover standard semantic execution and native collection.

Checklist

  • Formatted the code with Black.
  • Reviewed and updated the affected documentation and agent context.
  • Documented the public entry points in the API reference.
  • Ran focused behavioral checks.

matafela and others added 30 commits November 9, 2026 18:34
Order permission queries after the latest sensor sample on the caller's CUDA stream without resampling or advancing histories. Cover pending forbidden contacts and dropped evidence in standalone and history sampling paths.

Validation: 75 targeted tests passed; Black 26.10.1, public API documentation coverage and agent context checks passed.
Share Default CPU numeric observations within bounded command-gather scopes, reconstruct immutable standard targets, and transfer fresh internal command ownership while preserving public snapshots and validation.

Keep native multi-chunk grasp geometry and reductions resident on CUDA within each query, preserving samples, thresholds, stream ordering and independent results. Add focused regression coverage and a reproducible runtime comparison driver.

Validation: 201 focused tests including CUDA and 42 context tests passed; Black 26.10.1, API docs 4924/4924 and diff checks passed. Sources match the final benchmark manifest.

Forward/reverse sensor-free UR5 execution versus main 0119171 (seconds): small main 10.34-10.45, off 9.07-9.31, on 8.31-9.08; medium main 49.24-57.12, off 47.19-50.90, on 48.66-49.57; large main 251.77-260.14, off 189.57-196.42, on 192.07-200.29. All 492 program attempts accepted. Default CPU physics retains legacy CUDA grasp classification; these shared-host single-seed results are not physical qualification.
Restore the existing Default CUDA backend policy. Remove the newly introduced public selector and its ordinary-reporting test variant, retaining CPU contact permission coverage and documenting Direct GPU static-identity limits.

Validation: 53 configuration/contact tests including CUDA and 42 context tests passed. Black 26.10.1, API docs and diff checks passed.
Preserve legacy catalog identities when phase paths are inactive, while keeping explicit phase declarations in registration fingerprints. Align preview error coverage with aggregate input support and update state test doubles for Direct GPU queries, including publication guard coverage.

Validation: reproduced all 11 failed CI cases; 65 focused tests passed; broader Task Program and command/context suite 327 passed, 3 GPU cases skipped. Black 26.10.1, API docs 4963/4963, context map and diff checks passed. Existing context already describes inactive-default fingerprint compatibility; no context update required. Main synchronized through 438e111.
Move the existing pre-Scene collection after clearing facade-owned data and entity references. Cyclic wrappers newly orphaned by those clears must finalize while their Scene and World parents are still alive, instead of being collected during a later teardown.

Extend the existing deferred-destruction test with a cyclic view finalizer; both registry-ownership cases fail before the change. Keep backend synchronization, ownership checks, physical validation and GC count unchanged.

Validation: 288 manager CPU tests passed; the full 149-node failed CI GPU shard passed 143 with 6 pre-existing capability skips on pinned DexSim 0.6.0rc2. Black 26.10.1, API docs 4963/4963, context and diff checks passed. Lifecycle context already requires child-before-parent cleanup and remains accurate. The reported native abort itself was not deterministic locally; this fixes the demonstrated lifetime gap at its cleanup boundary.
…ings

Identify explicit references after aggregate expansion; round contact holds up to the control grid and verify measured elapsed time; resolve the default-plane actor only when enabled. Retire expired warming attempts without proposal/credit and allow active peers to drain and export, including expiry during initial validation.

Reject incompatible expert command modes, ActionManager declarations and controlled-joint selections before constructing a World. Preserve component-owned resource paths when relocating deployment snapshots by reusing the component resolver's resource normalization.

Add focused regressions and extend real workload integration with an expired group alongside a peer that commits and exports a receipt. Validation: 106 expansion unit tests, 4 native integration tests on pinned DexSim 0.6.0rc2, and 42 context tests passed. Regressions reproduce all six review findings against 722e385. Black 26.10.1, API docs 4963/4963 and diff checks passed. Update workload guidance for action-schema compatibility and admission expiry.
Normalize ULP-scale error in exact-grid hold durations before ceiling. Measure hold elapsed time with bounded fsum reductions of actual physics callback intervals instead of subtracting accumulated World clocks. Preserve distinct samples, upward rounding of real off-grid durations, both-finger contact and drift requirements.

Compare control_parts and active_joint_ids from resolved environment configurations rather than their shared implementation class. Replace artificial-class coverage with real decoded configurations and a full deployment-file regression sharing ordinary EmbodiedEnv.

Validation: 117 expansion unit tests, four native integration tests on pinned DexSim 0.6.0rc2 and 42 context tests passed. Regressions reproduce all three follow-up findings on 8c244cd, including 50 Hz/100 Hz boundary holds, 500k/5M-step clock accumulation, truly short windows and pre-World schema rejection. Black 26.10.1, API docs 5010/5010, context map and diff checks passed. Update context for measured hold timing and configuration-owned joint selections.
Keep landing-pose acceptance independent of the raised release target, covering zero and default clearance as well as configured lowerer ownership. Reconstruct the coordinated-transport configuration through its normal validator instead of replacing its normalized binding dictionary.

Validation: reproduced both failed CI cases; all 62 GenSim workflow tests and the native placement/release integration probe passed on pinned DexSim 0.6.0rc2. Black 26.10.1, API docs 5010/5010, context map and diff checks passed. No production behavior changed; existing GenSim context already describes clearance and landing validation correctly.
Bind the test validator to its measured environment and retain the real MeasuredTaskMonitor with deterministic joint evidence. This keeps hold-duration and contact assertions intact while exercising the new batched predicate boundary, instead of supplying an incomplete synthetic monitor.

Validation: 471 related expansion/bridge/configuration/GenSim tests passed, plus both native validation/selected-handover probes on pinned DexSim 0.6.0rc2. Black 26.10.1, API docs 5013/5013, context map and diff checks passed. Dataset cache was isolated to task-owned tmpfs because the host root filesystem had no non-reserved free space. Production behavior and documentation contracts are unchanged.
yuecideng and others added 25 commits November 17, 2026 05:26
Keep settling rows controller-active without trajectory or receipt credit. Retain configured grasp preload through coordinated placement. Use explicitly supported preceding-seed path kernels and cooperatively merge compatible requests, preserving active planning budgets and legacy solver fallbacks.

Validation: actual reset/grasp negative regressions reproduce on the published SDK; selected-row real path batching passes, motion checks 123 passed, workload integration 3 passed, SSIK checks 2 passed. Five-task repeated physical qualification follows separately.
Keep W times B physical capacity and original attempt/time/byte/step limits. Competing retries reserve only running slots; pending writes and confirmed commits retain the hard quota. Cancel peers through the original command handshake after confirmed persistence, retaining canonical outcomes and measured partial evidence.

Preserve phase metadata on safe stops and do not advance an already pending rejection hold twice. Native Task Program jobs enable idle-row reuse; custom sources opt in with an explicit cancellation port.

Validation: 41 focused lifecycle, configuration and real simulation checks pass on DexSim rc2. The actual known-bad scheduler copy fails the new unexecuted-hold regression. Independent engine/robotics/verification passes completed; API/context/Black gates pass. Full five-task after measurements follow on an installed VCS package.
A finished runtime may still have its last ordinary motion awaiting env.step. Keep the default fail-closed abort contract, but allow the joint controller owner to supply actual applied target positions when no runtime safe hold exists. Never reuse the pending motion or acknowledge before real physics consumption.

Retain the interrupted phase when terminal fields are empty, and keep the last actual scalar controller metadata and row identity in cancelled diagnostics without per-step serialization. Validation: 12 focused abort, safe-hold and actual reuse cases pass on rc2; full bottle trigger and repeated five-task qualification follow on the installed revision. Independent reviews completed.
Resolve scene preparation with the same precedence as EmbodiedEnv: explicit config first, registered defaults second. Normalize registered declarations through an owned FunctorCfg before shared-World compatibility checks.

Extend real deployment coverage for mismatched and identical registration defaults, explicit overrides, and equivalent explicit/registered declarations. Preserve single-workload validation behavior and document the effective-hook contract.
Generate layouts from authored anchors or workspace samples and alternate proposal directions within the existing budget. Retain the default bounded sampler, full task screening and independent measured acceptance. Compare all four strategies on pick/place, stacking and dual-arm RGB sorting.
Share simulator-free layout and feasibility declaration validation. Preserve final parent follower poses, distinguish sampled rejection from invalid authored anchors, retire exhausted warming rows, and record event transport history.
@han-xudong han-xudong added enhancement Improvements to existing capabilities; use feature for new capabilities. motion gen Robot motion planning, MotionGenerator, trajectory generation, retiming and expansion. atomic-skill Atomic Skill goals, planning, execution, verification and recovery. task-program Task Program language, semantics, compiler, runtime and integration contracts. labels Nov 19, 2026
Base automatically changed from codex/expansion-workloads to main November 19, 2026 09:46

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

atomic-skill Atomic Skill goals, planning, execution, verification and recovery. enhancement Improvements to existing capabilities; use feature for new capabilities. motion gen Robot motion planning, MotionGenerator, trajectory generation, retiming and expansion. task-program Task Program language, semantics, compiler, runtime and integration contracts.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants