Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions agent_context/topics/atomic-actions/atomic-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,35 @@ repeated-pick/open-drawer environment components carry the same Newton-only
overlays, while their declared Default backend continues to consume only the
portable fields.

For UR5 with `dexuni`, the paper-cup `move_held_object.py` tutorial opts into
VBD contact history with `latest` collision matching, sliding friction 4 on the
gripper alone, and a 0.0115 m candidate close target. Transport orientation
is not yet repeatably aligned. The native contact alpha and the cup
and ground materials are unchanged. `create_tutorial_simulation` accepts
`newton_solver_options`, and `add_tutorial_robot` accepts
`newton_gripper_friction` and optional `arm_damping`; these are authored before
model construction. This
calibration also applies to the UR5-only `pour.py` tutorial with its 0.013 m
cube close target. The UR5 plastic-tray `coordinated_pickment.py` tutorial uses
the same contact settings with a 0.023 m close target; the dual-arm helper
`add_dual_tutorial_robot` also accepts `newton_gripper_friction`.
The UR5 vertical-can `hand_over.py` tutorial uses those contact settings with
left/right close targets of 0.007/0.0055 m and retains its original arm drives.
Its replay callback holds the current targets for one second immediately before
the compiled `receive_release` segment so the receiving arm can settle before
opening. The wait uses segment metadata, not a fixed waypoint index, and is
scoped to that same profile. This scene-specific calibration does not promise
identical final poses or stability across initial-state changes. Horizontal
objects and other robot families retain their prior configuration.
The UR5 DexUni `place.py` profile uses a 0.013 m close target, gripper friction 4,
contact history with `latest` matching, contact alpha 0.5, and arm damping 1000.
The lower arm damping reduces tracking lag when the fingers open; finger
drives and authored release waypoints remain unchanged. Complete cached
replays preserve the grasp to release, but landing orientation still varies;
this is a scoped improvement, not a claim of full solver equivalence.
These explicit scene choices do not change world defaults
or establish a generic object-width-to-gripper-target mapping.

## Semantic integration boundary

`semantics` contains:
Expand Down
7 changes: 7 additions & 0 deletions agent_context/topics/sensor-system/lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,13 @@ support. Do not infer force support from sensor construction alone. Inspect
backend/solver/device matrix: Newton AutoSolver may change the answer during
preparation, so the manager checks again afterwards.

DexUni's `pure_mujoco` path reports rigid-contact geometry, normal impulse and
tangential impulse through the MuJoCo contact buffer, except when MuJoCo uses
its CPU implementation. Other DexUni paths expose available rigid-contact
geometry with `impulse=False` and `friction=False`; zero-valued output columns
there are placeholders, not measured zero forces. Particle-shape contacts do
not enter this rigid ContactQuery.

The sensor retains backend-emitted rows rather than reducing an actor pair to
one contact. Force-reporting paths apply the query's positive-impulse filtering;
geometry-only paths may provide valid rows with zero impulse. Solver settings
Expand Down
11 changes: 11 additions & 0 deletions agent_context/topics/simulation-system/articulation-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,21 @@ the native articulation before Direct GPU initialization and initial reset.
Iteration counts are configured together because the native setter is atomic.
Applying them only during facade binding is too late for captured GPU settings.

`root_props.newton_gravity_compensation` is Newton creation intent applied in
both asset physics modes; omission preserves the source/default policy. It
compensates articulation link weight without removing payload gravity or
contact reactions. DexSim checks the resolved solver's MuJoCo support.

`joint_drive_props` is the sole joint-property entry point; sparse fields retain
source values. `drive_type` describes the response and `target_mode` the active
command components. Lowering masks inactive gains: none/effort clear both,
velocity clears position gain. Newton rejects active acceleration drives;
non-MuJoCo position mode is gain-based emulation with zero velocity target.
Exact field options and defaults remain in `cfg/articulation.py`.
Newton-specific `joint_drive_props.passive_damping` lowers to passive DOF
damping, separately from the drive's velocity gain. It retains the same
exact-name/regex/control-part matching and remains active for passive or effort
target modes. Applied Newton-only settings are rejected on Default.

## State order and batched writes

Expand Down Expand Up @@ -122,3 +131,5 @@ FREE-joint synchronization, mesh export and no-op root writes;
`tests/sim/objects/test_articulation.py` for topology/FK. Source configuration
and root timing are covered by `tests/sim/spawn/test_descriptors.py` and
`tests/sim/test_sim_manager.py`.
`tests/sim/test_dexuni_two_way.py` compares contact-limited dynamic fingers
against the same one-way commands, including CUDA Graph and reset.
17 changes: 13 additions & 4 deletions agent_context/topics/simulation-system/physics-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Return to the [simulation overview](simulation-system.md) for lifecycle entry.
## Backend, solver and device resolution

`SimulationManagerCfg.physics_cfg` selects the backend by config type.
`cfg/simulation.py` owns `PhysicsBackendCfg`, `DefaultPhysicsCfg` and
`cfg/physics.py` owns `PhysicsBackendCfg`, `DefaultPhysicsCfg` and
`NewtonPhysicsCfg`; do not add another backend string to the manager config.
Gym's file-owned selector and composition rules belong to
[environment configuration](../env-framework/configuration.md).
Expand All @@ -26,7 +26,7 @@ The concrete solver is resolved from the complete Spawn scene, then exposed
through `NewtonPhysicsBackend.solver_type`. Do not guess a solver-specific robot
preset while it is unresolved. Gradient mode requires an explicit
`semi_implicit` solver. Read exact solver options/defaults in
`cfg/simulation.py` and the installed DexSim config types; do not substitute a
`cfg/physics.py` and the installed DexSim config types; do not substitute a
hardcoded fallback when an expected DexSim API is absent.

`collision_cfg` owns the external Newton contact pipeline. Its
Expand All @@ -38,10 +38,19 @@ external-pipeline tuning does not configure internal contact generation.
The manual differentiable trajectory requires the external pipeline; see
[differentiable environment](../differentiable-env/differentiable-env.md).

DexUni preserves its one-way contact policy unless `coupling="two_way"` is
explicit. Two-way feedback requires dynamically integrated MuJoCo joints and
rejects full-surface rigid-soft contacts. Prescribed kinematic trajectories
cannot respond to contact reactions. Forward solver expert options through
`NewtonPhysicsCfg.solver_cfg`; DexSim owns their validation and dispatch.
Articulation gravity compensation and passive damping are separate creation
settings; see [articulation adapters](articulation-adapters.md).

Capability checks remain explicit: Default owns native rigid constraints and
Newton deformables are separate. Contact support is rechecked after Newton
solver resolution; DexUni and MuJoCo-Warp on CPU do not expose the supported
ContactQuery path. See `physics/` and [sensors](../sensor-system/sensor-system.md).
solver resolution; capabilities depend on the concrete DexUni contact path
and whether MuJoCo runs on CUDA. See `physics/` and the
[contact capability contract](../sensor-system/lifecycle.md#contact-queries).

## Sparse source physics

Expand Down
42 changes: 42 additions & 0 deletions docs/source/api_reference/embodichain/embodichain.lab.sim.cfg.rst
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,13 @@ For manager-owned Newton steps, ``sync_to_renderer=None`` or ``False`` publishes
only for visual consumers; ``True`` additionally requests publication on steps
without consumers.

DexUni accepts ``coupling="two_way"`` with ``joint_mode="dynamic"`` to feed
contact reactions back to articulations. Keep ``collision_cfg=None`` when
DexUni owns collision detection. ``coupling_options`` configures the native
feedback policy; full-surface rigid-soft contacts are unsupported in this mode.
Articulation gravity compensation and passive joint damping are configured
separately below.

.. currentmodule:: embodichain.lab.sim.cfg.physics

.. autosummary::
Expand All @@ -148,6 +155,41 @@ without consumers.
physics_backend_from_cfg
validate_physics_cfg

Articulation configuration module
---------------------------------

Articulation types live in ``embodichain.lab.sim.cfg.articulation`` and remain
available from the ``embodichain.lab.sim.cfg`` facade. Root creation settings
are independent of sparse link and joint physics overlays.
``root_props.newton_gravity_compensation`` compensates articulation link
weight on MuJoCo-backed Newton solvers while retaining payload gravity and
contact reactions. Omission preserves source intent.

``NewtonJointDrivePropertiesCfg.passive_damping`` selects passive DOF damping
through the same exact-name, regex and control-part rules as the drive gains.
It is independent of the drive velocity gain ``damping`` and stays active for
passive and effort target modes. In dictionaries, select this subtype with
``joint_drive_props.backend="newton"``. Both settings require Newton.

.. currentmodule:: embodichain.lab.sim.cfg.articulation

.. autosummary::

ArticulationRootPropertiesCfg
LinkPhysicsOverrideCfg
link_attrs_from_dict
JointDrivePropertiesCfg
NewtonJointDrivePropertiesCfg
ArticulationCfg

.. autoclass:: ArticulationRootPropertiesCfg
:members:
:no-index:

.. autoclass:: NewtonJointDrivePropertiesCfg
:members:
:no-index:

Compatibility facade
--------------------

Expand Down
36 changes: 33 additions & 3 deletions embodichain/lab/sim/cfg/articulation.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,15 @@
_rigid_body_physics_from_dict,
)

__all__ = [
"ArticulationRootPropertiesCfg",
"LinkPhysicsOverrideCfg",
"link_attrs_from_dict",
"JointDrivePropertiesCfg",
"NewtonJointDrivePropertiesCfg",
"ArticulationCfg",
]


def _normalize_joint_target_mode(value: object) -> int:
"""Normalize a portable joint target mode to its backend integer value."""
Expand Down Expand Up @@ -94,6 +103,16 @@ class ArticulationRootPropertiesCfg:
min_velocity_iters: int | None = None
"""Default-only minimum root velocity-solver iterations (0 to 255)."""

newton_gravity_compensation: float | None = None
"""MuJoCo link-weight compensation fraction in [0, 1], on Newton only.

``None`` preserves the source/default policy; zero retains gravity and one
compensates every link in this articulation. Payload gravity and contact
reactions remain active. This creation setting applies in both asset
physics modes, independently of DexUni's coupling policy. The finalized
Newton solver must support MuJoCo gravity compensation.
"""

def __post_init__(self) -> None:
"""Require the two values consumed by the atomic Default setter."""
if (self.min_position_iters is None) != (self.min_velocity_iters is None):
Expand Down Expand Up @@ -376,12 +395,23 @@ def to_dict(self) -> dict[str, Any]:

@configclass
class NewtonJointDrivePropertiesCfg(JointDrivePropertiesCfg):
"""Compatibility subtype for serialized Newton joint-drive configs.
"""Joint-drive configuration with Newton-specific passive dynamics.

``target_mode`` is now portable and lives on
:class:`JointDrivePropertiesCfg`. The subtype remains so existing
``backend="newton"`` dictionaries and round trips retain their type; new
robot definitions should use the common class.
``backend="newton"`` dictionaries and round trips retain their type. Use
the common class when no Newton-specific dynamics are required.
"""

passive_damping: dict[str, float] | float | None = None
"""Nonnegative passive joint damping, independent of drive damping.

Uses the same joint-name, regex and robot control-part rules as the drive
gains. Apply it with ``asset_physics_mode='overlay'``; ``None`` preserves
source damping. Unlike :attr:`damping`, this
lowers to Newton's ``JointDofConfig.damping`` and remains active when the
drive target mode is ``none`` or ``effort``. It requires Newton and is
consumed by MuJoCo-backed solvers, including dynamic DexUni.
"""


Expand Down
8 changes: 8 additions & 0 deletions embodichain/lab/sim/cfg/physics.py
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,14 @@ class NewtonPhysicsCfg(PhysicsBackendCfg):
parameters accepted by that DexSim solver config. If omitted, EmbodiChain
preserves DexSim's scene-aware ``AutoSolverCfg`` default. A DexSim build
exporting ``AutoSolverCfg`` is required; no concrete-solver fallback is used.
For ``solver_type='dexuni'``, the current DexSim contract keeps expert
options under ``vbd_options``, ``mujoco_options`` and
``collision_options``; use ``joint_mode`` and ``contact_mode`` for the
coupled dispatch policy. ``coupling='two_way'`` enables contact reactions
on dynamically integrated articulations. Optional ``coupling_options``
accepts ``mass_scale``, ``proxy_relaxation`` and ``iterations``. Two-way
coupling requires ``joint_mode='dynamic'`` and does not support full-surface
rigid-soft contacts; DexSim validates these combinations.
"""

collision_cfg: NewtonCollisionPipelineCfg | Mapping[str, Any] | None = field(
Expand Down
2 changes: 2 additions & 0 deletions embodichain/lab/sim/cfg/robot.py
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,8 @@ def resolve(
solver_candidates = []
if solver_type != "auto":
solver_candidates.append(f"newton_{solver_type}")
if solver_type == "dexuni":
solver_candidates.extend(("newton_mujoco_warp", "newton_mjwarp"))
if solver_type == "mujoco_warp":
solver_candidates.append("newton_mjwarp")
candidates = (*solver_candidates, "newton", "default")
Expand Down
2 changes: 1 addition & 1 deletion embodichain/lab/sim/objects/articulation.py
Original file line number Diff line number Diff line change
Expand Up @@ -1045,7 +1045,7 @@ def root_link_name(self) -> str:
Returns:
str: The name of the root link.
"""
return self.entities[0].get_root_link_name()
return self._entities[0].get_root_link_name()

@cached_property
def joint_names(self) -> List[str]:
Expand Down
4 changes: 2 additions & 2 deletions embodichain/lab/sim/objects/backends/collision.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,9 +184,9 @@ def _from_descriptor(entity: object) -> list[CollisionShapeDesc] | None:
dtype=torch.float32,
).reshape(3)
shapes: list[CollisionShapeDesc] = []
# Match native shape queries: disabling contact response keeps geometry
# available to planners, including moving obstacles that exert no impulses.
for shape_idx, collision in enumerate(collisions):
if getattr(collision, "enable_collision", True) is False:
continue
geometry_type = getattr(collision, "geometry_type", None)
geometry_type = getattr(geometry_type, "value", geometry_type)
local_pose = torch.as_tensor(
Expand Down
15 changes: 13 additions & 2 deletions embodichain/lab/sim/objects/backends/newton.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,11 +204,14 @@ def _configure_newton_mimic_compliance(
backend = get_newton_backend(result.world)
if (
backend is None
or backend.solver_type != "mujoco_warp"
or backend.solver_type not in {"mujoco_warp", "dexuni"}
or backend.cfg.requires_grad
or (backend.model is not None and backend.model.requires_grad)
):
return False
solver = backend.mujoco_solver
if solver is None:
return False

relation_names = [
(state_joint_names[child_id], state_joint_names[parent_id])
Expand Down Expand Up @@ -284,13 +287,21 @@ def _configure_newton_mimic_compliance(
f"Newton model has no mimic constraint for joint pair {error.args[0]}."
) from error

solver = runtime.solver
mapping = getattr(solver, "mjc_eq_to_newton_mimic", None)
mjw_model = getattr(solver, "mjw_model", None)
if mapping is None or mjw_model is None:
raise RuntimeError("MuJoCo-Warp did not materialize Newton mimic rows.")

mapping_values = np.asarray(mapping.numpy())
if backend.solver_type == "dexuni":
from newton import Model

scene_rows = backend.get_mujoco_index_map(
Model.AttributeFrequency.CONSTRAINT_MIMIC
).numpy()
mapping_values = mapping_values.copy()
valid = mapping_values >= 0
mapping_values[valid] = scene_rows[mapping_values[valid]]
selected = np.isin(mapping_values, constraint_rows)
if int(selected.sum()) != len(constraint_rows):
raise RuntimeError(
Expand Down
28 changes: 21 additions & 7 deletions embodichain/lab/sim/physics/newton.py
Original file line number Diff line number Diff line change
Expand Up @@ -201,19 +201,33 @@ def supports_rigid_object_group(self) -> bool:
@property
def supports_contact_sensor(self) -> bool:
# ContactSensor consumes the backend-neutral Scene ContactQuery API,
# but not every Newton solver/device pair currently publishes that
# query. Keep this runtime-sensitive so ``solver_type='auto'`` can be
# rechecked after Spawn finalization resolves the concrete solver.
# but CPU MuJoCo cannot publish its force report. Keep this runtime-
# sensitive so ``auto`` and DexUni's scene-selected solver path are
# rechecked after Spawn finalization.
solver_type = self.solver_type
if solver_type is None:
return True
solver_type = solver_type.lower()
if solver_type.endswith("dexuni"):
return False
if solver_type == "dexuni":
world = getattr(self._manager, "_world", None)
if world is None:
return True
from dexsim.engine.newton_physics.backend_registry import (
get_newton_backend,
)

backend = get_newton_backend(world)
solver = None if backend is None else backend.solver
features = None if solver is None else getattr(solver, "features", None)
if getattr(features, "backend", None) == "pure_mujoco":
mujoco_solver = backend.mujoco_solver
return mujoco_solver is not None and not mujoco_solver.use_mujoco_cpu
# Other DexUni paths expose rigid-contact geometry through
# ContactQuery, which marks impulse and friction unavailable.
return True
runtime_device = self._runtime_device
if runtime_device is None:
runtime_device = str(getattr(self._manager, "device", ""))
return not (
solver_type.endswith("mujoco_warp")
and runtime_device.lower().startswith("cpu")
solver_type == "mujoco_warp" and runtime_device.lower().startswith("cpu")
)
4 changes: 3 additions & 1 deletion embodichain/lab/sim/sensors/contact_sensor.py
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,9 @@ class ContactSensor(BaseSensor):

The sensor preserves every backend-emitted contact row that passes the
backend's positive-impulse filter. Geometry-only Newton solvers retain all
candidate contact rows.
candidate rigid-contact rows. Inspect :attr:`contact_capabilities` before
interpreting ``impulse`` or ``friction``: DexUni paths other than
``pure_mujoco`` do not report either quantity.
"""

SUPPORTED_DATA_TYPES = [
Expand Down
Loading
Loading