Repository navigation
Integrate cables with Gym observations, replay and visualization - #839
Conversation
Migrate DexUni example settings to the current namespaced solver interface and preserve the existing cloth contact values. Update the Newton config regression test and document the new mapping contract.
Port the retained MJVBD V2 integration consumers to the current DexUni public interface, while keeping AutoSolver as the Newton default. Preserve namespaced solver options, tutorial calibration, reset/mimic handling, and current task configuration.
Allow geometry-only ContactQuery data for DexUni VBD paths and full impulse/friction data for pure MuJoCo. Add capability tests and GPU end-to-end coverage.
…e-adaptation # Conflicts: # embodichain/lab/sim/cfg/simulation.py # tests/sim/test_cfg.py
Synchronize main at 6881882 with the open environment's DexSim 0.6.0+c3e92584 build. Read solver contact reports through the new API, retaining per-row impulse validity alongside main's contact cache, observer lifecycle and fused loss counters. Forward raw MuJoCo contact parameters through sparse configuration and document the geometry-detection/report distinction. Require dexsim_engine>=0.6.0,<0.7.0. Validation: 3474 focused CPU regressions passed; 17 CUDA sampling tests passed; 2 native DexUni coupling tests passed; 10 native contact/support/camera checks passed (the support fetch hook was updated and rerun separately). Black 26.10.1: 2215 files unchanged. API docs: 4286/4286. Context checks and tests passed. All 146 referenced DexSim import symbols resolved. Wheel built and verified against worktree bytes and installed DexSim version. Hybrid headless tests used SDL_VIDEODRIVER=dummy; shared conda packages were not reinstalled. The complete example scene matrix and interactive windows were not rerun.
Keep gravity compensation and opt-in mimic stabilization as the supported Newton extensions. Remove seven unused root fields, generic SDK option forwarding and articulation-wide SDF rewriting. Leave native creation/geometry choices at source or SDK defaults and update the owning context. Reuse existing configuration, source overlay and mimic coverage; reject withdrawn fields at both public decode boundaries and roundtrip non-default values for retained fields. Remove obsolete capability-forwarding cases. Validation on conda open / DexSim 0.6: 724 CPU tests passed, 7 deselected; 2 native two-way coupling tests passed with and without CUDA Graphs. Black 26.10.1: 2215 files unchanged; API docs 4286/4286; context checks/tests and final wheel/source metadata validation passed.
Use the current DexUni configuration contract directly. Remove AutoSolver aliases, legacy flat-option translation and attribute injection, SDK-version probing, and stale solver guards. Preserve native default routing, grouped expert options and contact_mode=auto. Update configuration regression coverage and project context. Validation: 728 focused CPU tests passed, 20 deselected; black 26.10.1 left 2215 files unchanged; API docs cover 4286/4286 exports; context validation and routing checks pass. The rebuilt wheel matches the changed source and accepts the installed DexSim 0.6.0+c3e92584 build. Native GPU selections were not rerun for this configuration cleanup.
Replace retired solver terminology throughout the tracked documentation and tests. Document the default DexUni family separately from scene ownership and contact_mode=auto, migrate the example's iteration options into vbd_options, and correct path-specific contact-report guidance. Cover the explicit automatic contact policy and native ownership defaults using existing configuration cases. Validation: 738 focused CPU/docs regressions passed, 20 deselected; nine Python snippets parse, and updated Python/YAML configurations convert to native DexUni automatic defaults. Black 26.10.1, API documentation coverage (4286/4286), context validation and whitespace checks pass. A tracked-file scan finds no retired solver names. The Sphinx build is retried after this commit because the architecture generator requires committed source.
Keep reset-time contact observations readable until the next solved report, require eager native acquisition outside caller-owned CUDA capture, and normalize Newton impulses with the internal solver duration while retaining World-step history clocks. Cover real articulation resets, native graph consumers, multi-substep support forces, variable World intervals and history clocks; update sensor guidance and align timeline reset test buffers with quantity validity.
485dfd5 to
6ca6566
Compare
612b68d to
0e304ee
Compare
[Medium impact] The PR changes cable behavior across Gym recording, replay, and browser viewing. Fix inline cable visual loading before merging.
|
#839 录制回放与 Gym 修改方案(6ca65663)沿用 #838 的 DexUni 整场 episode reset。建议将轨迹坐标契约与 Gym 配置修复作为本层合入门槛。 合入前必须修复
后续优化Viser 先在设备端筛选可见环境,再传输姿态。 流体导出已有这个顺序。按 1,024 环境、每条 128 段计算,每帧全部姿态约 3.5 MiB,只看一个环境所需约 3.5 KiB。建议在扩大并行可视化规模前处理。验收时选择单个非零环境及非连续多环境,检查送入 CPU 转换的段数只随所选行增长,输出位置只应用一次原点偏移。帧耗时收益需另行实测。 |
0e304ee to
91ae175
Compare
Synchronize through fcca082 while retaining cable Gym observations, replay and Viser integration on the current simulation APIs. Preserve main's cable lifecycle fixes and published runtime configuration.
yuecideng
left a comment
There was a problem hiding this comment.
Reviewed the 25-file delta from fcca082 to 5d8f201 with independent architecture, engine/HPC and API passes. Two P2 findings remain, covering per-environment cable replay and visible-environment export scaling. The inline mesh configuration regression is fixed. Focused tests passed, including the existing two-environment CUDA recording check on this commit. The saved-artifact reproduction uses production CPU recording/save/load APIs. Export timings measure the CPU conversion phase and exclude native fetching and actual CUDA transfer. Recommend addressing the findings before merge.
| ) | ||
| for key, expected in initial.items(include_nested=True, leaves_only=True): | ||
| observed = current[key].to(expected) | ||
| if observed.shape != expected.shape or not torch.allclose( |
There was a problem hiding this comment.
[P2] Preserve source arena placement for per-environment cable exports
With record_trajectory=True and the default trajectory_auto_save=True, a batched scene saves each row via _save_trajectory_for_env() -> save_trajectory(env_ids=[env_id]). Each artifact declares num_envs=1 while retaining that row's cable world positions. For source arena 1, an otherwise matching single-environment replay resets at arena 0 and this comparison rejects it. Replaying the artifact in the original batch also fails the environment-count guard at lines 113-116. meta.env_ids is saved but never used to resolve placement. A production save/load CPU probe accepts row 0 at X [0.1, 0.2, 0.3], but rejects row 1 at X [5.1, 5.2, 5.3]; both files contain raw_observations. The CUDA recording test disables auto-save, so it misses this default output path. Persist source arena origins or store relative positions, then compare the initial states in a common frame while retaining the full-reset/solver-history restriction. Cover auto-saving a nonzero row and replaying the resulting file in a fresh single-env deployment.
There was a problem hiding this comment.
Fixed in 6a7a29d. Cable exports now retain arena_origins_w in selected-row order. Replay translates the captured reset positions into the source frame for comparison, while preserving raw world observations. Coverage includes source rows 0/1, changed target origins, legacy files and real mismatches. The native regression now uses public reset with auto-save enabled for arena 1, loads the actual file into a fresh single-env scene and verifies the final state after action replay.
| raise RuntimeError( | ||
| "Call SimulationManager.prepare() before capturing cable state." | ||
| ) | ||
| pose = data.segment_pose_w |
There was a problem hiding this comment.
[P2] Select visible cable rows before converting poses to CPU
This returns the full CUDA (B,S,7) batch, and pose_to_position_wxyz() copies and normalizes all rows before the subsequent env_ids selection. Even the default env_ids=[0] therefore incurs work proportional to every simulated cable. A transfer-boundary probe for B=4096/S=256 receives 29,360,128 pose bytes while publishing 256 nodes needing 7,168 bytes. On a Ryzen 9 9950X / NumPy 2.2.6, the actual CPU conversion-plus-selection phase measured 49.21 ms median versus 0.015 ms for equivalent selected-first conversion, with identical outputs (two warmups, seven repetitions). These timings exclude native fetching and actual CUDA transfer. Other asset paths already follow this order in main, but this new branch extends that cost to cable scenes. Gather the selected environment/segment rows on device before conversion. Add a regression that checks the shape at the CPU boundary for a nonzero or noncontiguous visible selection.
There was a problem hiding this comment.
Fixed in 6a7a29d. Pose batches now gather visible environment/component rows on their source device before CPU conversion. The regression verifies the boundary shape and noncontiguous row order for NumPy, Torch CPU and CUDA. On the RTX 5090, the 4096x256 transfer-plus-conversion median decreased from 30.39 ms to 0.0573 ms with equal outputs; the 16x64 case remained approximately 0.05 ms. Measurements exclude native state fetching and browser publication. Full workload and timing details are recorded in the PR description.
Integrate main at 7b9f55b while preserving cable and fluid configuration, recording and replay paths. Both replay suites and native cable recording pass.
Description
Integrate
CableObjectwith Gym scene construction, the unified ObservationManager, trajectory recording and Viser through the DexUni runtime. Cable observations carry world XYZ, XYZW, COM linear velocity and angular velocity for every segment. Shared history retains T+1 state boundaries for T actions.Per-environment cable exports now save
meta.arena_origins_win exported row order. Dynamic replay translates captured reset positions into the recorded world frame for initial validation, so a trajectory automatically saved from arena 1 can replay in a fresh single-environment scene. Raw observations retain their world coordinates. Files without origin metadata retain the existing matching-world-placement requirement. Replay still requires matching environment counts and full-reset initial states.Viser gathers visible environment/component poses on their source device before CPU transfer and quaternion normalization. Segment capsules retain their original world placement, orientation and node order. Gym accepts both file-backed and inline cable visual meshes.
Stack
main, synchronized through7b9f55b836da7d9795e51259ae6ff2006f661872.Review fixes
Validation
Latest main synchronization is committed at
60046dc577ff558d725f27fb3a5d2dac7a2120ad, integrating main at7b9f55b836da7d9795e51259ae6ff2006f661872. Conflict resolution preserves both cable and fluid configuration, recording and replay paths.6a7a29d5dalso exercised the CUDA visible-row exporter and confirmed that new regression cases fail before the fixes. Existing legacy replay, T+1 storage, missing-cable and genuine reset-mismatch checks remain covered.Runtime checks use Python 3.12, Torch 2.11.0+cu130 and local DexSim build
88ac9206e0cb9c7894df6c7780f3e157806f0239on an RTX 5090. Runtime dependency metadata follows main.Export measurement
One visible environment, float32 input, seed 42, two warmups and seven repetitions. These CUDA timings include pose gathering, transfer to CPU and quaternion conversion, with Torch CUDA synchronization around timing. Native state fetching and browser publication are outside the measurement. Outputs match exactly.
Hardware/software: Ryzen 9 9950X, RTX 5090, NumPy 2.2.6, Torch 2.11.0+cu130 and 16 Torch CPU threads. Native batch fetching remains owned by the object adapter.
Type of change
Example
Checklist
black .with Black 26.10.1.