Repository navigation
Adapt DexSim 0.6 and consolidate DexUni two-way support - #829
Conversation
Migrate DexUni example settings to the current namespaced solver interface and preserve the existing cloth contact values. Update the Newton config regression test and document the new mapping contract.
Port the retained MJVBD V2 integration consumers to the current DexUni public interface, while keeping AutoSolver as the Newton default. Preserve namespaced solver options, tutorial calibration, reset/mimic handling, and current task configuration.
Allow geometry-only ContactQuery data for DexUni VBD paths and full impulse/friction data for pure MuJoCo. Add capability tests and GPU end-to-end coverage.
…e-adaptation # Conflicts: # embodichain/lab/sim/cfg/simulation.py # tests/sim/test_cfg.py
Merge the direct/named SimpleSolverProvider integration onto PR #829 while preserving DexUni defaults, grouped solver options and strict name validation. Use the current extension capability contract, including explicit particle, control and native acceleration support. Retain position holds for capable solvers and support effort-only initialization and selective reset. Validate named discovery with isolated distribution metadata so the conda open installation remains unchanged. Document running this worktree via PYTHONPATH. Validation (conda open, DexSim 0.6.0+c3e92584, Newton 1.7.0, Warp 1.17.0): - Provider/config/backend helper tests: 84 passed, including CPU/CUDA parity, gradients, named/direct factories and native selective resets. - Articulation/config/backend regression: 142 passed, 35 GPU cases skipped. - CPU and CUDA two-environment demos: native/functional parity and gradients. - Context tests: 42 passed; context validation and API docs coverage passed. - Black 26.10.1 and diff whitespace checks passed. Context: revise physics configuration ownership for extension selection and functional dynamics; robot preset selection remains unchanged.
han-xudong
left a comment
There was a problem hiding this comment.
Reviewed 8f792d08 on an RTX 4080 with a clean Python 3.12.3 environment, Torch 2.11.0+cu128, TorchVision 0.26.0+cu128, TorchCodec 0.11.1+cu128, and the DexForce DexSim 0.6.0rc1 wheel. The environment imported CUDA 12.8, LeRobot 0.6.1, Newton 1.7.0, Warp 1.17.0, and MuJoCo/MuJoCo Warp 3.15.0. pip check reported No broken requirements found.
Clean cu128 environment and test commands
Run from the EmbodiChain checkout at 8f792d08.
python3.12 -m venv .venv
source .venv/bin/activate
dexsim_wheel=dexsim_engine-0.6.0rc1-cp312-cp312-manylinux_2_31_x86_64.whl
curl -fLO "http://pyp.open3dv.site:2345/packages/$dexsim_wheel"
echo "cfa15734b9c7e9217719de24c8a807b7650fe24506129270cb62016040669b61 $dexsim_wheel" | sha256sum -c
uv pip install --python .venv/bin/python --torch-backend cu128 \
--default-index https://pypi.org/simple \
--index http://pyp.open3dv.site:2345/simple/ \
--allow-insecure-host pyp.open3dv.site:2345 --index-strategy unsafe-best-match \
'torch @ https://download-r2.pytorch.org/whl/cu128/torch-2.11.0%2Bcu128-cp312-cp312-manylinux_2_28_x86_64.whl#sha256=d252cf975fb18c94a85336323ad425f473df56dab35a44b00399bd70c7a3b997' \
'torchvision @ https://download-r2.pytorch.org/whl/cu128/torchvision-0.26.0%2Bcu128-cp312-cp312-manylinux_2_28_x86_64.whl#sha256=ccf26b4b659cfce6f2208cb8326071d51c70219a34856dfdf468d1e19af52c0d' \
'torchcodec @ https://download-r2.pytorch.org/whl/cu128/torchcodec-0.11.1%2Bcu128-cp312-cp312-manylinux_2_28_x86_64.whl#sha256=74d692c90a5b0fbb71a6d100b792a19b0b4731c2aeb81b52cd523eb634c0c80c' \
'numpy==2.2.6' 'tensordict==0.15.3' 'mujoco==3.15.0' 'mujoco-warp==3.15.0' -e . "./$dexsim_wheel"
python -m pip check
python -c 'import dexsim, torch; print(dexsim.__version__, torch.__version__, torch.version.cuda)'
export SDL_VIDEODRIVER=dummy EMBODICHAIN_SIM_EXIT_PROCESS=0 PYTEST_DISABLE_PLUGIN_AUTOLOAD=1
export PYTHONPATH="$PWD" OMP_NUM_THREADS=2 OPENBLAS_NUM_THREADS=2 MKL_NUM_THREADS=2
python -m pytest --run-gpu -q -ra \
tests/sim/sensors/test_contact_newton_e2e.py \
tests/sim/sensors/test_support_observation_native.py
python -m pytest --run-gpu -q -ra \
tests/sim/sensors/test_contact_query_sensor.py \
tests/sim/sensors/test_contact_history.py \
tests/sim/sensors/test_contact_sensor_graph.pyThe native contact and support suite passed 6 tests. The contact query, history, and Graph suite passed 55 tests. The two DexUni coupling tests also passed, covering the simulator's internal graph with capture on and off.
Standalone repro for the three contact findings
Save the following as repro.py and run each mode in a fresh process with the environment above.
python repro.py graph
python repro.py reset
python repro.py force 1
python repro.py force 10Observed results
graphfails in the native snapshot'swait_eventduring capture withdependency created on uncaptured work in another stream.resetreads successfully before reset, raisesContact state is unavailable after reset until the next World.updateimmediately after reset, and reads successfully after one more step.force 1reports native 9.810 N, history 9.810 N, and a threshold hit.force 10reports native 9.810 N, history 0.981 N, and a missed threshold.
import sys
from contextlib import nullcontext
from importlib.metadata import version
from pathlib import Path
from tempfile import TemporaryDirectory
from types import SimpleNamespace
import torch
from dexsim.engine.newton_physics import teardown_newton_physics
from dexsim.engine.newton_physics.backend_registry import get_newton_backend
from newton import Contacts
from embodichain.lab.gym.envs.base_env import BaseEnv
from embodichain.lab.sim.cfg import (
ArticulationCfg, MassPropertiesCfg, NewtonPhysicsCfg, RenderCfg,
RigidBodyPhysicsCfg, RigidObjectCfg,
)
from embodichain.lab.sim.sensors import ContactSensorCfg
from embodichain.lab.sim.shapes import CubeCfg
from embodichain.lab.sim.sim_manager import SimulationManager, SimulationManagerCfg
SLIDER = """<robot name="slider">
<link name="base"/>
<link name="moving">
<inertial><mass value="0.1"/>
<inertia ixx="0.001" ixy="0" ixz="0" iyy="0.001" iyz="0" izz="0.001"/>
</inertial>
<collision><geometry><box size="0.05 0.05 0.05"/></geometry></collision>
</link>
<joint name="slide" type="prismatic">
<parent link="base"/><child link="moving"/>
<origin xyz="0 0 0.5"/><axis xyz="1 0 0"/>
<limit lower="0" upper="0.1" effort="10" velocity="1"/>
</joint>
</robot>"""
def check_graph(sim, sensor):
sim.update(step=150)
stream = torch.cuda.Stream(device=sim.device)
stream.wait_stream(torch.cuda.current_stream(sim.device))
with torch.cuda.stream(stream):
sensor.update()
assert sensor.get_data()["is_valid"].any().item()
stream.synchronize()
graph = torch.cuda.CUDAGraph()
with torch.cuda.graph(graph, stream=stream):
sensor.update()
print("Capture completed")
def check_reset(sim, sensor):
env = SimpleNamespace(
sim=sim, sensors={"contacts": sensor}, device=sim.device,
num_envs=sim.num_envs, _camera_group_ids=[],
_profiler=SimpleNamespace(section=lambda *_: nullcontext()),
)
assert not sensor.requires_substep_update
sim.update(step=3)
BaseEnv._get_sensor_obs(env)
print("Before reset: observation succeeds")
sim.reset_objects_state(env_ids=[0], excluded_sensor_uids=[sensor.uid])
sensor.reset(env_ids=[0])
try:
BaseEnv._get_sensor_obs(env)
except RuntimeError as error:
print(f"Immediately after reset: {error}")
else:
print("Immediately after reset: observation succeeds")
sim.update(step=1)
BaseEnv._get_sensor_obs(env)
print("After one more physics step: observation succeeds")
def check_force(sim, sensor, substeps):
sim.update(step=150)
history = sensor.create_history(
"cube-force", sensor.get_actor_ids("cube"),
counterpart_ids=sensor.get_actor_ids("ground"), force_threshold=1.0,
)
sensor.begin_control_step()
sim.update(step=1, after_substep=sensor.update_physics_step)
backend = get_newton_backend(sim._world)
assert backend.solver.features.backend == "pure_mujoco"
report = Contacts(
backend.mujoco_solver.get_max_contact_count(), 0,
device=backend.model.device, requested_attributes={"force"},
)
backend.mujoco_solver.update_contacts(report, backend.state_0)
count = int(report.rigid_contact_count.numpy()[0])
force = report.force.numpy()[:count, :3]
normals = report.rigid_contact_normal.numpy()[:count]
native_n = abs((force * normals).sum(axis=1)).sum()
history_n = torch.linalg.vector_norm(history.force).item()
print(f"{substeps} substeps: native={native_n:.3f} N, history={history_n:.3f} N, "
f"threshold_hit={history.found.item()}")
def main(mode, urdf, substeps):
sim = SimulationManager(SimulationManagerCfg(
headless=True, device="cuda:0",
render_cfg=RenderCfg(renderer="auto" if mode == "force" else "no-render"),
physics_cfg=NewtonPhysicsCfg(
device="cuda:0", physics_dt=0.01, num_substeps=substeps,
use_cuda_graph=False,
),
))
try:
if mode == "reset":
sim.add_articulation(ArticulationCfg(
uid="slider", fpath=str(urdf), build_pk_chain=False,
))
if mode == "force":
sim.add_rigid_object(RigidObjectCfg(
uid="ground", shape=CubeCfg(size=(1.0, 1.0, 0.1)),
attrs=RigidBodyPhysicsCfg(), body_type="static", init_pos=(0.0, 0.0, 0.10),
))
sim.add_rigid_object(RigidObjectCfg(
uid="cube", shape=CubeCfg(size=(0.1 if mode == "reset" else 0.2,) * 3),
attrs=RigidBodyPhysicsCfg(mass_props=MassPropertiesCfg(mass=1.0)),
init_pos=(0.0, 0.0, 0.2 if mode == "reset" else 0.4),
))
sensor = sim.add_sensor(ContactSensorCfg(
uid="contacts", rigid_uid_list=["ground", "cube"] if mode == "force" else ["cube"],
filter_need_both_actor=mode == "force", max_contacts_per_env=32,
))
if mode == "graph":
check_graph(sim, sensor)
elif mode == "reset":
check_reset(sim, sensor)
else:
check_force(sim, sensor, substeps)
finally:
sim.destroy(exit_process=False)
SimulationManager.flush_cleanup_queue()
if __name__ == "__main__":
assert version("dexsim-engine") == "0.6.0rc1"
assert torch.version.cuda == "12.8" and torch.cuda.is_available()
mode = sys.argv[1]
assert mode in ("graph", "reset", "force")
substeps = int(sys.argv[2]) if mode == "force" else (10 if mode == "reset" else 1)
with TemporaryDirectory() as directory:
urdf = Path(directory) / "slider.urdf"
urdf.write_text(SLIDER, encoding="utf-8")
try:
main(mode, urdf, substeps)
finally:
teardown_newton_physics()Keep reset-time contact observations readable until the next solved report, require eager native acquisition outside caller-owned CUDA capture, and normalize Newton impulses with the internal solver duration while retaining World-step history clocks. Cover real articulation resets, native graph consumers, multi-substep support forces, variable World intervals and history clocks; update sensor guidance and align timeline reset test buffers with quantity validity.
Synchronize origin/main at 460939c. Resolve deformable configuration, Spawn descriptor, example, metadata and context conflicts while retaining explicit tetrahedral meshes, cloth extensions and the native contact sampling fixes. Validation in conda open: 737 focused regressions, 147 integration regressions and 15 native physics regressions passed. Black 26.10.1 leaves all 2301 files unchanged; API coverage is 4440/4440 and context validation passes.
[High impact] The latest changes appear safe to merge; no new actionable issues were found. SummaryThe PR adapts simulation support to DexSim 0.6 and consolidates DexUni configuration. The latest changes improve container downloads and installation guidance.
No new actionable issues were found. This review did not run builds or runtime tests. Reviews (4) · Last reviewed commit: "Merge concurrent fluid update and retain..." · Reviewed by Greptile |
Check original tetrahedral and surface indices before int32 narrowing. Declare the native GLB pan frame in all placement suites and propagate it through grasp, target and footprint qualification. Require complete footprint coverage, nonpenetrating sampled gaps and an interior-floor support witness for measured placement, including curved undersides. Align two-way contact guidance and resolve the Task Engine bundle fixture's execution-policy component selector through the production resolver. Cover overflow, geometry failure cases and the real GLB loader in Default and Newton.
Document Python 3.11/3.12 support and the actual py311 CI activation. Distinguish the current rc1 profiles from historical 0.5.1 images. Retry the official fixed-release Miniforge alias after a versioned-URL failure while retaining the original SHA256 check. Validated both published rc1 ABI wheels and profile hashes, core-wheel Python-version acceptance/rejection, and 45 release/workflow regressions. Container build qualification follows separately.
Co-authored-by: kioxli <162790769+kioxli@users.noreply.github.com>
Record published cp311/cp312 wheel hashes, Python-version dry runs, unchanged locked resolutions, successful public/internal container builds and final pip checks. Keep image promotion and full native runtime acceptance separate from these build/dependency results.
Preserve origin/codex/adapt-latest-dexsim at e14fc4e while integrating the completed Python distribution and container dependency follow-up. Dependency and Docker build inputs remain identical to the qualified runtime snapshot. Post-integration: 46 release/workflow tests pass, Black leaves 2312 files unchanged, API coverage is 4449/4449 and context validation passes.
han-xudong
left a comment
There was a problem hiding this comment.
CI follow-up for c6e991c2: please fix the reset-test fixture issue below, retain all existing assertions, and attach successful focused Gym/reset and fluid regressions plus full CI on the final head before merging. The build passed, but the test gate failed.
Merge order: #829 → #838 → #839 → #834 → #923. #829 can land independently of the downstream #834 fix once its own validation passes.
Match SimulationManager's fluid registry in the SimpleNamespace reset fixture. Preserve every reset count, canonical row ID, sensor exclusion and row-isolation assertion. Reproduced all five AttributeError failures before the change. Validation in conda open: 165 focused Gym/reset and fluid regressions plus 5 native fluid lifecycle/visibility regressions passed without skips.
Description
Complete the DexSim 0.6 adaptation on top of main
1e75b963f. The branch retains main's exactdexsim_engine==0.6.0rc1requirement and Python >=3.11 metadata.Review fixes
support_mesh_frame: native_glbin all three placement benchmark suites. The shared grasp, target and geometry audit use the native GLB's local-Z handle axis; the example's authored OBJ keeps its local-Y convention. Existing suite asset scales and root poses are retained.Validation
Runtime: conda
open, RTX 5090, Python 3.11.14, Torch 2.7.1+cu128, Warp 1.17.0, Newton 1.7.0, installed DexSim0.6.0+c3e92584. The behavioral tests below qualify that development build. Published rc1 wheels additionally passed the installation and container-dependency follow-up below; the complete robot demonstration was not requalified.test_static_deployment_roundtrip_without_live_resources, reproduced locally before the fix and passes afterwards.black==26.10.1 black .: 2301 files unchanged. API coverage: 4440/4440 exports documented. Context validation passes.Python installation and Docker dependency follow-up
Completed the two historical P1 follow-ups. Recorded evidence distinguishes the new checks from the old 0.5.1 image/native qualification.
c1af874bae7f0a6bc53691a8b2b9d2e063d80e57. It adds an official same-release Miniforge alias fallback, with the original SHA256 enforced, after the versioned download returned HTTP 504.e14fc4e78is preserved. Its dependency/Docker inputs are unchanged from the qualified snapshot; post-integration release/workflow checks passed 46 tests, Black left 2312 files unchanged, API coverage passed 4449/4449, and context validation passed.These are distribution, resolution, build and installed-dependency checks. The OCI candidates were exported locally. Registry promotion and the full GPU/media/Blender behavior suite were not performed; the added fluid runtime is outside this qualification.
Main synchronization and CI follow-up (2026-11-10)
Merged main
1070fcdc4and retained the DexSim, fluid and placement fixes while adopting main's Atomic Skill package/API names. Resolved the example/test conflicts and updated imports introduced by this PR.The broader CPU run exposed two regressions inherited from main: the pose-reset camera fixture lacked the new ground-visibility lifecycle state, and
camera_ground_visibilityomitted the common Viser configuration. The fixture now mirrors camera initialization, and the example accepts the shared--viser*options and passes their configuration to the manager. Existing coverage was reused; no new example/tutorial test cases were added.The previous test job exceeded GitHub's six-hour limit after reporting fluid failures and stopping in the rigid-object module. Native resource lanes now stop on the first failure, print each node and use short tracebacks plus a 120-second faulthandler dump, so a later native hang cannot hide an earlier failure. Discovery and ordered-collection validation are preserved.
Local validation on RTX 5090 with the published
dexsim_engine==0.6.0rc1wheels:black==26.10.1 black .: 2314 files unchanged. Public API coverage: 4683/4683. Context-map validation passed. The merged runtime context remains accurate; the CI diagnostic change needs no new runtime guidance.Current ground-visibility CI repair
Merged main
1e75b963f, adopting its updated installation guide and NVIDIA/DLSS requirements while retaining this branch's container qualification record and the workflow's actualpy311activation.The preceding remote test job passed 13,208 CPU tests and all five fluid integration cases, then failed the ground-visibility RGB assertion. Its images were black after
NVSDK_NGX_VULKAN_GetCapabilityParametersfailed (0xbad00007) on the RTX 3090 runner. The case labeled FastRT was also using the process's previously initialized Hybrid renderer.The existing two ground-visibility cases now execute in separate native processes and assert the actual renderer enum. They explicitly select un-denoised window/offscreen rendering so their raw RGB/depth contract does not depend on DLSS/NGX or reconstruction at 32×32. All original image-difference, depth, selective-reset, metadata, physical-floor and topology-rebuild assertions remain intact.
Validation with the published Python 3.11 rc1 wheel on RTX 5090:
Current remote validation: CI run 42793115761, head
a8a85a006. Lint, test planning, docs-changes and documentation tests passed. After more than 44 minutes of monitoring, build/test were still queued while both shared Linux runners were busy; the complete remote result has not yet been confirmed.