Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion agent_context/MAP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ topics:
aliases: [simulation system, simulation manager, Newton physics backend, motion module, sim motion, camera attachment, 运动能力模块, 仿真系统, 仿真管理器]
keywords: [SimulationManager, SimulationManagerCfg, RenderCfg, DLSSCfg, DLSS, frame_time_delta_ms,
offscreen_dlss_enabled, explicit physics stepping, arena, DexSim, PhysicsBackendCfg,
DefaultPhysicsCfg, NewtonPhysicsCfg, AutoSolverCfg, DexUniSolverCfg, DexUni, runtime control, deformable, particle set,
DefaultPhysicsCfg, NewtonPhysicsCfg, AutoSolverCfg, DexUniSolverCfg, DexUni, MJVBDV2SolverCfg, mjvbd_v2, mjvbd2, runtime control, deformable, particle set,
ArticulationJointKinematics, get_parent_joint_chain, asset_physics_mode, joint_drive_props, enable_gravity,
fixed-cadence playback, quaternion, xyzw]
paths: [topics/simulation-system/simulation-system.md]
Expand Down
29 changes: 29 additions & 0 deletions agent_context/topics/atomic-actions/atomic-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,35 @@ repeated-pick/open-drawer environment components carry the same Newton-only
overlays, while their declared Default backend continues to consume only the
portable fields.

For UR5 with `mjvbd_v2`, the paper-cup `move_held_object.py` tutorial opts into
VBD contact history with `latest` collision matching, sliding friction 4 on the
gripper alone, and a 0.0115 m candidate close target. Transport orientation
is not yet repeatably aligned. The native contact alpha and the cup
and ground materials are unchanged. `create_tutorial_simulation` accepts
`newton_solver_options`, and `add_tutorial_robot` accepts
`newton_gripper_friction` and optional `arm_damping`; these are authored before
model construction. This
calibration also applies to the UR5-only `pour.py` tutorial with its 0.013 m
cube close target. The UR5 plastic-tray `coordinated_pickment.py` tutorial uses
the same contact settings with a 0.023 m close target; the dual-arm helper
`add_dual_tutorial_robot` also accepts `newton_gripper_friction`.
The UR5 vertical-can `hand_over.py` tutorial uses those contact settings with
left/right close targets of 0.007/0.0055 m and retains its original arm drives.
Its replay callback holds the current targets for one second immediately before
the compiled `receive_release` segment so the receiving arm can settle before
opening. The wait uses segment metadata, not a fixed waypoint index, and is
scoped to that same profile. This scene-specific calibration does not promise
identical final poses or stability across initial-state changes. Horizontal
objects and other robot families retain their prior configuration.
The UR5 V2 `place.py` profile uses a 0.013 m close target, gripper friction 4,
contact history with `latest` matching, contact alpha 0.5, and arm damping 1000.
The lower arm damping reduces tracking lag when the fingers open; finger
drives and authored release waypoints remain unchanged. Complete cached
replays preserve the grasp to release, but landing orientation still varies;
this is a scoped improvement, not a claim of full solver equivalence.
These explicit scene choices do not change world defaults
or establish a generic object-width-to-gripper-target mapping.

## Semantic integration boundary

`semantics` contains:
Expand Down
5 changes: 4 additions & 1 deletion agent_context/topics/motion-planning/collision-worlds.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@ mapping key is the authoritative logical/source obstacle ID used by the
content-cache key, `collision_world_entity_ids`, generated obstacle-name prefix,
dynamic update key, and registry validation. Scene generation reads physical
collision descriptors through `RigidObject.get_collision_shapes()` and emits a
mixed tensor-backed cuRobo scene. A compound source expands to physical names
mixed tensor-backed cuRobo scene. Geometry snapshots retain authored collision
shapes even when physical contact response is disabled, matching native shape
queries; a moving planner obstacle can therefore avoid exerting impulses on
the robot. A compound source expands to physical names
such as `registry_id__shape_0`; cache and full-world identity remain keyed by
the unexpanded `registry_id`, and dynamic updates fan out through the physical
shapes' local poses. A registry mapping whose source lacks physical collision
Expand Down
146 changes: 112 additions & 34 deletions agent_context/topics/simulation-system/simulation-system.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,8 @@ before closing the Spawn result, environment, and World. Default backends use
the no-op hook. Newton synchronizes its resolved Warp CUDA device and clears
its render bridge while Spawn still owns the parent skeletons, so cached link
views cannot be destructed after their native parents.
Asset material and reset-baseline material caches are also cleared before
closing the Scene, so OptiX material wrappers cannot outlive the renderer.

After backend materialization, dynamic `RigidObject`, `Articulation`, and
`RigidObjectGroup` facades capture their resolved mass, inertia diagonal, and
Expand Down Expand Up @@ -466,18 +468,59 @@ are both absent. Thus a config-backed launcher preserves the backend default,
while an explicit CPU override remains authoritative even for Newton.
Environment tensors derive from `sim.device`; do not introduce a second
tensor-device selector.
Leaving `NewtonPhysicsCfg.solver_cfg=None` preserves DexSim's
`AutoSolverCfg` default. A DexSim build exporting `AutoSolverCfg` is required;
EmbodiChain does not substitute a concrete solver. DexSim resolves that
placeholder from the complete Spawn scene during finalization: rigid-only
scenes select XPBD, scenes with an articulation select MuJoCo Warp, and
supported particle families select their matching particle/deformable solver.
A mapping with `solver_type: auto` or
`class_type: AutoSolverCfg` is the explicit equivalent. Gradient mode must
still select `semi_implicit` explicitly because AutoSolver does not choose a
differentiable solver. Before finalization, EmbodiChain treats `auto` as
unresolved; after finalization, `NewtonPhysicsBackend.solver_type` reads the
concrete type from DexSim's World-owned backend.
`NewtonPhysicsCfg()` selects `mjvbd_v2` by default for rigid bodies and
articulations. V2 owns collision detection, so conversion omits the external
pipeline even when `collision_cfg` contains its default settings. Other
solvers retain the external pipeline unless explicitly disabled.
Explicit `solver_cfg=None`, `solver_type: auto`, or `class_type: AutoSolverCfg`
opts into DexSim's scene-aware selection: rigid-only scenes select XPBD,
articulation scenes select MuJoCo Warp, and supported particle families select
their matching solver. Gradient mode still requires explicit `semi_implicit`.
Before finalization, explicit `auto` is unresolved; after finalization,
`NewtonPhysicsBackend.solver_type` reads the concrete World-owned solver.
Explicit rigid/articulation V2 configurations select `mjvbd_v2` (aliases
`mjvbd2`, `mjvbdv2`, or `MJVBDV2SolverCfg`). Keep `joint_mode: dynamic` for
ordinary robot control, put existing MuJoCo options under `mujoco_options`,
and set `collision_cfg: null` because V2 owns collision detection. Joint
targets, state access, mimic compliance, and whole-world articulation reset
use the existing Scene/Batch path. V2's compact MuJoCo indices are translated
by DexSim for contact queries and by its index map for mimic equality rows.
Runtime FK updates only MuJoCo-owned articulation trees; replaying FK for
VBD-owned FREE joints would overwrite simulated rigid-body poses with stale
generalized coordinates. Reset completeness checks count Scene articulations,
not the internal FREE-joint trees used to represent standalone rigid bodies.
The pure MuJoCo branch exposes contact geometry, impulses, and friction on
CUDA. VBD contacts expose geometry without MuJoCo impulse reporting.

V2 selects MuJoCo for articulations with active scalar joints and VBD for
unselected free rigid bodies. Its current mixed dynamic branch is one-way:
VBD contacts do not push back on the MuJoCo robot. The local, uncommitted DexSim
patch enables a second MuJoCo-Warp collision/contact pipeline by default; explicitly setting
`mujoco_options.disable_contacts=True` still disables them. The compact
MuJoCo view contains owned links and world-attached static shapes, but excludes
VBD-owned free bodies and independent kinematic bodies. A stationary fixture
declared kinematic therefore does not provide robot contact reaction in this
view; use static for fixtures that never move. The stationary tables in
`examples/sim/demo/grasp_cup_to_caffe.py` and
`scripts/tutorials/atomic_action/coordinated_placement.py` use this declaration.
Moving kinematic obstacles and
dynamic-object reaction remain separate limitations. Mixed contact queries
still lack equivalent impulse reporting, even when robot ground support is
restored. Explicit `mujoco_articulations` or
`mujoco_articulation_paths` can select complete joint trees; these are ownership
choices, not automatic EmbodiChain rewrites. This integration does not extend
deformable support.

The UR5 `repeated_pick_place` and `open_drawer` deployments provide
`task.ur5.newton.yaml` and matching `env.newton.yaml` components using V2. They share the
existing embodiment, controller gains, and Task Programs, with the timed
`trajectory_open_loop_slow.yaml` execution policy (80 control points). The
40-point drawer reference allocates only five samples to pulling; at 25 Hz its
0.2-second pull is shorter than the time needed for 18 cm at the drawer's
authored 0.35 m/s speed limit. Evaluate measured lift, placement, and drawer
travel separately from projected Task Program completion; a completed program
alone is not evidence of physical success.

Explicit coupled articulation/deformable configurations use `dexuni` or
`DexUniSolverCfg`. DexUni owns collision detection and its particle-shape
contacts, so explicit configurations set `collision_cfg=None` instead of
Expand All @@ -491,28 +534,43 @@ The `open_drawer.py` tutorial combines this option with 20 Newton substeps per
10 ms control step, while keeping its authored robot gains, collision geometry,
pull trajectory, success criteria, and push trajectory identical to Default.
Atomic-action tutorials configure their shared Newton simulation in
`scripts/tutorials/atomic_action/tutorial_utils.py` with 20 solver substeps
per 10 ms physics step (0.5 ms solver intervals). They select `mujoco_warp` with
`use_mujoco_contacts=True` and set the external `collision_cfg` to `None`.
MuJoCo-Warp then generates and solves contacts internally at every solver
substep; external-pipeline settings such as `update_interval`, contact
reduction, and the external `rigid_contact_max` do not apply. DexSim derives
the native per-world contact capacity from the finalized scene, avoiding the
oversized fixed buffers formerly inherited from external-pipeline examples.
The shared factory leaves the Default backend configuration unchanged. For
parallel-jaw manipulation, the tutorial and packaged task components add a
Newton-only contact overlay (`condim=4`, `ke=4e4`, `kd=4e2`, torsional friction
`0.1`, rolling friction `0.01`) to the gripper and directly manipulated object
contact links. `condim=4` activates torsional friction in MuJoCo-Warp; rolling
friction is retained for solvers/`condim=6` that support it. This is a scoped
contact/force-closure candidate profile, not a replacement for Newton's scene-aware
`AutoSolverCfg`; rigid-only scenes still resolve to XPBD and articulation
scenes to MuJoCo-Warp. Keep
backend-specific overlays sparse so the Default adapter continues to use its
existing portable contact envelope.
The package dependency must identify the exact DexSim dev build containing
this API; a base `==0.4.3` requirement also accepts older local-version wheels
that do not export `AutoSolverCfg` and is therefore insufficient.
`scripts/tutorials/atomic_action/tutorial_utils.py` with `mjvbd_v2`, retaining
20 solver substeps per 10 ms physics step (0.5 ms solver intervals) and the
existing articulation tuning under `mujoco_options`. The direct drawer
tutorial and rigid manipulation examples use the same native V2 ownership.
The shared factory leaves the Default backend configuration unchanged.
Parallel-jaw manipulation keeps its Newton-only contact overlay (`condim=4`,
`ke=4e4`, `kd=4e2`, torsional friction `0.1`, rolling friction `0.01`) on the
gripper and manipulated objects. MuJoCo options affect articulation-owned
contacts; free rigid-body contacts use VBD and need not have identical effects.
The UR5 `pickup.py` and `axis_align.py` tutorials use a 0.013 m
DH-gripper close target for their 5 cm cube under V2
(`MJVBD_V2_CUBE_CLOSE_QPOS` in `tutorial_utils.py`). The old
0.036 m target relies on contact feedback to stop the fingers; V2's one-way
proxies instead reach that target and squeeze the cube out. This calibration
is specific to that object and gripper, not a general grasp-width mapping.
These two cube tutorials retain native VBD defaults. Their calibrated target
does not make the original oversized command compatible with one-way contact.
The UR5 `move_held_object.py` paper-cup tutorial uses a 0.0115 m candidate close target,
gripper-only sliding friction 4, VBD contact history, and collision matching
`latest`, retaining the native contact alpha. The shared simulation helper
accepts scene-specific `newton_solver_options`; robot creation accepts
`newton_gripper_friction` before spawning. The cup candidate has not achieved repeatable orientation alignment. These are explicit tutorial choices,
not world defaults, and leave the cup and ground materials unchanged. The UR5-only `pour.py` tutorial uses the same contact history and gripper
friction with a 0.013 m cube close target; its V2 initial height includes the
ground's 1 mm contact margin. Other robot selections and physics solvers keep
their existing targets and materials.
The UR5 V2 `place.py` tutorial additionally uses contact alpha 0.5 and arm
damping 1000 with the same cube target, gripper friction, and contact history.
`add_tutorial_robot(arm_damping=...)` authors this drive override before spawn;
other callers retain their configured gains. Release waypoints are unchanged,
and landing orientation remains sensitive to contact variation.
The `examples/sim/demo/scoop_ice.py` scene sets
`vbd_options.rigid_body_contact_buffer_size=256` for dense ice contacts.
This per-body VBD capacity is independent of MuJoCo's `nconmax`/`njmax`;
overflow truncates the contacts used by each affected VBD body. The larger
scene-specific capacity does not establish stable friction grasping.
Explicit cloth, softbody, and differentiable examples keep their selected solvers.
Newton's `suppress_warp_kernel_logs=True` suppresses Warp's one-time runtime
banner plus module compile/load chatter during manager startup, build, facade
initialization, and physics updates, then restores the process-wide setting.
Expand All @@ -521,6 +579,10 @@ information is controlled separately by `dexsim_startup_info`.

EmbodiChain-authored Newton collision shapes use a default margin and gap of
`0.001 m` each only when no portable or Newton-native envelope is authored.
The Pour tutorial starts its cube center at half its height. Under V2, Pour
and Place add 1 mm initial clearance for the default ground's contact margin,
avoiding penetration recovery before grasp planning. The global ground
configuration stays unchanged to preserve the pure-MuJoCo locomotion path.
`CollisionPropertiesCfg.contact_offset/rest_offset` are portable: Default uses
them directly, while the Spawn compiler maps `rest_offset → margin` and
`contact_offset - rest_offset → gap`. Both values must be present to derive a
Expand Down Expand Up @@ -721,6 +783,10 @@ only the generic `newton` and `default` alternatives are eligible; do not guess
a solver-specific preset before DexSim has inspected the complete scene. This
is the only robot preset selection boundary; do not add a second backend
selector to robot configs.
For explicit `mjvbd_v2`, preset priority is `newton_mjvbd_v2`, then
`newton_mujoco_warp` / `newton_mjwarp`, then generic `newton` / `default`.
This preserves established articulation drive tuning while allowing a V2
override.

File-backed rigid objects and articulations share one source-independent
physics policy: `asset_physics_mode="preserve"` keeps properties resolved from
Expand Down Expand Up @@ -1021,3 +1087,15 @@ Default source-physics capture preserves each link's replication collision
filter while filling mass/COM properties. Spawn articulation mass-property
writes use the live handle's `set_link_inertia` and `set_link_com_pose`; native
COM quaternions are converted from EmbodiChain `xyzw` to DexSim `wxyz`.

## Current MJVBD V2 validation dependency

The local mixed-scene validation uses an uncommitted DexSim patch that defaults
`mujoco_options.disable_contacts` to `False` while retaining
`use_mujoco_contacts=True`. This runs MuJoCo-Warp collision/contact solving for
its articulation/static compact view alongside Newton/VBD contacts. It differs
from native V2 mixed-mode contact configuration and is not an accepted final
architecture. Mixed locomotion alignment measurements depend on this patch;
they do not establish native single-pipeline mixed-mode equivalence. A second
local DexSim patch fixes convex mesh import coordinates. Neither patch is
contained in this EmbodiChain commit.
4 changes: 3 additions & 1 deletion embodichain/lab/sim/cfg/robot.py
Original file line number Diff line number Diff line change
Expand Up @@ -349,14 +349,16 @@ def resolve(
solver_type = str(
solver_cfg.get("solver_type")
or solver_cfg.get("class_type")
or "auto"
or "mjvbd_v2"
)
else:
solver_type = str(getattr(solver_cfg, "solver_type"))
solver_type = _normalize_newton_solver_type(solver_type)
solver_candidates = []
if solver_type != "auto":
solver_candidates.append(f"newton_{solver_type}")
if solver_type == "mjvbd_v2":
solver_candidates.extend(("newton_mujoco_warp", "newton_mjwarp"))
if solver_type == "mujoco_warp":
solver_candidates.append("newton_mjwarp")
candidates = (*solver_candidates, "newton", "default")
Expand Down
Loading