Repository navigation
Description
Proposal
Introduce a reusable actuator integration in which Newton supplies model computation, DexSim owns simulation integration and lifecycle, and EmbodiChain supplies user configuration and robot/task composition.
Start with explicit PD and DC-motor torque-speed saturation. Expose one SDK-owned execution path usable by standalone DexSim scenes and EmbodiChain environments, then qualify additional backends and model types incrementally.
This is a separate follow-up to #836. Keep the existing fused motor implementation in that PR; this proposal is not a merge prerequisite or a request to enable the motor in current task defaults.
Motivation
- The motor introduced by Reduce RL rollout overhead and align batched physics updates #836 combines reusable actuator mathematics with scene-specific joint binding, native force buffers, and runtime scheduling. Moving the reusable integration to the SDK gives standalone and Gym callers the same lifecycle.
- Actuator state must participate in selective episode reset, topology rebuilds, graph capture, and teardown. The missing reset hook identified during review illustrates why this ownership needs to be explicit.
- Newton already provides composable actuator models. Reusing those components reduces duplicated PD, saturation, delay, and learned-drive implementations.
- EmbodiChain should retain convenient model selection, named control groups, robot presets, command access, and reward telemetry without managing a second native execution loop.
- Default/PhysX and Newton may eventually share explicit model computation, while retaining their own state/command adapters and native implicit-drive behavior.
Additional context
1. Current implementation and scope
At #836 head 75e4466e6a4f56999ff2e842d6ee2147942aa46b:
_BatchedJointDCMotorControlderives from DexSimGraphRuntimeControl. It resolves scalar joints, computes PD plus DC saturation, writes native force buffers, and publishes effort/velocity buffers.SimulationManager._register_joint_dc_motor_control()registers it before scene preparation.- It evaluates only at
substep_index == 0and holds effort through the remaining solver substeps. - The reset review finding concerns the native
Scene.reset_arenas()path. Ordinary manager resets take a different route; this issue must not be described as an existing failure of every RL reset.
At #837 head 209bd6931789c1bb02d589554bdbe526fad11de7, IsaacVelocityEnv has an optional registration path, but UnitreeG1IsaacRoughEnv.explicit_pd_effort_control is currently False. The default G1 task therefore does not currently exercise this motor. Explicit-path qualification must enable it deliberately.
2. Responsibility boundaries
| Layer | Responsibility | Boundary |
|---|---|---|
| Newton | Reusable Actuator, DrivePD, clamping, delay, and other model components |
Compute effort from state and commands; expose model state and supported execution capabilities |
| DexSim | Actuator descriptors, backend lowering, finalized joint binding, command staging, force submission, scheduling, state ownership, reset/rebuild/capture/close | Own exactly one live actuator execution path per driven joint |
| EmbodiChain | Thin typed configuration and lowering, named control groups, robot/task model selection, public command and telemetry access | Use SDK APIs without owning native solver buffers or another stepping loop |
embodichain_robots |
Concrete model presets and calibrated actuator parameters | Keep vendor/model defaults separate from generic SDK implementation and task logic |
Newton actuators accept state/control array protocols and can be constructed outside ModelBuilder; reuse is not intrinsically limited to Newton physics scenes. Backend adapters still require their own qualification. See Newton 1.7 actuator documentation.
Isaac Lab provides a useful comparison, rather than a module layout to copy wholesale. Its current main documentation describes configuration and named groups above native execution: supported explicit models run through Newton or a shared PhysX/OVPhysX adapter. See Isaac Lab native actuators. These current upstream docs are architectural references, not a claim that the historical #836 benchmark used that implementation.
3. Configuration and parameter ownership
Introduce a minimal SDK descriptor and thin EmbodiChain config for:
- Articulation identity and resolved joint selection; preserve public joint order, custom control-part names, and distinct coordinate/DOF layouts.
- Model selection and its typed parameters: initially PD gains, actuator effort clipping, DC saturation effort, and the rated joint-side speed.
- Explicit execution cadence and hold behavior.
- Command components and telemetry semantics, with backend/solver capability validation.
Names such as ActuatorDesc or ActuatorCfg are provisional. Reuse existing public structures where they already own the required contract.
Keep the existing articulation configuration/compiler as the owner of joint-property lowering:
- Physical joint bounds, armature, friction, and solver constraints remain joint properties.
- Explicit actuator gains and model-side saturation belong to the actuator. Implicit gains remain solver-drive properties.
- Distinguish the actuator's torque-speed/effort limits from solver-enforced joint limits; they may legitimately have different values.
- For explicit ownership, lower native PD gains to zero/disabled as required so the actuator and solver do not both apply PD. Preserve authored configuration and reset baselines.
- Define source-preserve/overlay precedence and conversion of existing presets. Reject ambiguous duplicate authorship instead of maintaining two live copies of the same gain.
- Keep the no-actuator-config path compatible with existing implicit drives. Unsupported model/backend/solver combinations must fail before partial scene construction or command mutation.
Likely EmbodiChain integration points are cfg/articulation.py, spawn/descriptors.py, and the existing Scene object adapters. Concrete robot defaults stay in the bundled robot extension.
4. Command path and execution cadence
policy/controller command
-> EmbodiChain action and named-group mapping
-> DexSim command staging
-> scheduled actuator evaluation using Newton components
-> native effort submission
-> physics integration
-> SDK telemetry exposed through the EmbodiChain facade
The SDK must distinguish policy/control intervals, outer physics steps, and solver substeps. To migrate the current motor, preserve evaluation once at the beginning of each outer physics step and zero-order hold over its solver substeps.
For the referenced G1 configuration, these rates are 50 Hz policy, 200 Hz actuator/outer physics, and 400 Hz solver integration. They are an acceptance example, not global defaults. Moving evaluation to every solver substep would change the controller and must not happen implicitly.
Use DexSim's existing runtime-control/build integration as the initial seam. If a native solver-integrated actuator path is later selected, prove equivalent scheduling and ensure the same joints are not also executed by a runtime control.
5. Runtime contracts
Joint ownership and force submission
- Resolve joint names to finalized q/qd indices once per topology revision. Handle floating-base offsets and non-contiguous public selections without assuming q and qd share a layout.
- Validate scalar-DOF support for the first implementation and reject unsupported joints. Respect existing mimic coupling and passive joints.
- Reject overlapping actuator ownership and conflicts with exclusive runtime controls.
- Newton actuator output uses scatter-add. Define clearing and composition so effort does not accumulate across steps and unrelated joint/external contributions are preserved. Current motor overwrite behavior must be reproduced deliberately.
- Reuse device buffers and selectors on the steady-state path; avoid per-environment host loops, repeated state fetches, and implicit synchronization.
Lifecycle and state
- Declare through public SDK scene/articulation APIs before the relevant preparation boundary; tasks must not reach into
_physics_bindingor_spawn_scene.builder. - Let DexSim own the actuator instance, buffers, and any delay/PID/recurrent state. Reset selected arenas through the existing preflight/commit protocol, restoring the appropriate targets and clearing selected diagnostics/state while preserving unselected rows and captured buffer addresses.
- Connect both native Scene reset and EmbodiChain manager reset to the same SDK-owned actuator reset semantics, with one effective reset per selected episode boundary.
- Rebind after topology changes, invalidate stale borrowed views/graphs, release partial initialization resources, and close child resources before native parents.
- Future stateful models must account for state-buffer exchange and graph replay, including odd/even evaluation counts. Scope checkpoint support explicitly when those models are introduced.
Commands and telemetry
- Expose desired position/velocity/feedforward inputs and distinguish pre-clamp computed effort from post-clamp actuator output.
- Document that post-clamp output is an actuator command, not necessarily measured solver reaction force.
- Preserve the motor's sampled pre-integration velocity and last-evaluation timing needed by rewards/acceleration calculations.
- State whether returned buffers are borrowed or owned and when they become stale; retain efficient device views for RL consumers.
Capabilities and compatibility
- Validate graphability and differentiability for the selected components and backend. The current motor disables backward execution, so migration alone does not establish differentiable-training support.
- Preserve existing nonfinite behavior deliberately. CPU comparison during review found equivalent finite/overspeed outputs but different NaN handling between the fused motor and a direct Newton actuator composition.
- Keep Newton-specific imports and capabilities isolated so the existing Default path remains usable where those optional components are unavailable.
6. Backend and delivery plan
| Stage | Scope | Exit condition |
|---|---|---|
| 1. Contract and baseline | Pin supported SDK/Newton versions; define descriptors, parameter ownership, cadence, command/telemetry and lifecycle contracts | CPU regressions and representative baseline measurements identify current behavior |
| 2. DexSim Newton integration | Implement PD plus DC-motor composition, finalized binding, effort submission, selective reset, rebuild and capture integration | Standalone SDK scenes pass the explicit-path contract on declared supported solvers |
| 3. EmbodiChain facade and opt-in migration | Add thin config/lowering and command/telemetry access; migrate the private motor caller through SDK APIs | Equivalent outputs, cadence, resets and task behavior; current implicit defaults remain intact |
| 4. Default/PhysX adapter | Qualify reuse of the same explicit models through native state/command adapters | Backend parity tests and measured transfer/staging costs justify enabling supported combinations |
| Later extensions | Delay, PID, learned drives and validated gradient/checkpoint support | Each addition defines state, reset, graph and resource behavior before being advertised |
Stages may ship as focused SDK and EmbodiChain changes with explicit version dependencies. Default support and stateful models do not need to block the initial qualified Newton integration. Link the SDK implementation work from this issue when its repository/MR is selected.
7. Validation and acceptance criteria
- Pin the tested DexSim, Newton, Warp, solver, device, and execution modes.
- Reuse/extend
tests/sim/test_joint_motor.pyand the existing manager/reset tests; add SDK-owned lifecycle tests where the implementation lives. Use independent expected outputs or the established fused implementation as an explicit migration oracle. - Cover PD/DC saturation at rest, rated speed, overspeed braking, reordered public joints, floating-base q/qd offsets, untouched joints, and the agreed NaN/Inf behavior.
- Verify outer-step effort holding with multiple solver substeps and policy decimation; exercise the opt-in task path explicitly.
- Verify selective reset for non-contiguous arena selections through native Scene and manager entry points, preserving unselected state and graph buffer addresses.
- Verify topology rebuild, stale-view handling, construction failure cleanup, and teardown with appropriately small native fixtures.
- Verify raw desired targets, physical joint constraints, explicit/implicit gain ownership, effort composition, and telemetry timing through the production config/load path.
- Run CPU behavioral checks plus serial native GPU eager/CUDA Graph checks. State any unavailable backend/solver capabilities instead of treating skipped checks as passed.
- Benchmark the existing fused kernel against native actuator composition: small batches and a representative 4,096-environment/37-joint workload; report construction/capture/warmup separately from steady-state evaluation, memory, launch count, and end-to-end rollout throughput under identical settings.
- Explain measured performance tradeoffs. Multiple native component kernels are not evidence of either a material regression or a speedup; retain a specialized fast path only with demonstrated need and shared behavior tests.
- Update public API documentation and the owning project context when contracts change; verify installed-package imports and the supported dependency boundary.
- Keep Reduce RL rollout overhead and align batched physics updates #836's current implementation until a separate migration is qualified. Any later change to a task's default actuator selection requires its own behavioral evidence.
8. Related work
- Reduce RL rollout overhead and align batched physics updates #836: current fused motor and simulation APIs.
- Add G1 rough locomotion and USD-native dexterous manipulation RL tasks #837: optional reference locomotion integration; the current G1 default leaves explicit motor control disabled.
- [Proposal] Unify differentiable environments and APG training infrastructure #848: differentiable environment/APG infrastructure; relevant to later gradient support, not a prerequisite for the initial explicit-actuator integration.
Checklist
- I have checked that there is no similar issue in the repo (required). Searched existing issues for actuator, motor, NewtonActuator, and the Chinese actuator term; [Proposal] Unify differentiable environments and APG training infrastructure #848 is related but covers a different infrastructure scope.
Activity