Repository navigation
Align locomotion defaults with validated DexUni training - #1130
Merged
Merged
Conversation
acrlw
marked this pull request as ready for review
November 10, 2026 02:17
yuecideng
self-requested a review
November 11, 2026 02:18
yuecideng
approved these changes
Nov 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR makes the six official flat-ground velocity tasks default to the DexUni → CUDA MuJoCo Warp configuration used by the validated Unitree training runs. The Newton environments explicitly set the solver and its MuJoCo options, and their catalogs and README commands select the Newton deployment. Both backend configurations also preserve unbounded policy actions with
clip: null.The existing Newton YAMLs select MuJoCo Warp directly with 10 solver iterations. The validated runs instead use DexUni with dynamic joints, automatic contact selection, 20 solver iterations and 50 line-search iterations, with CPU MuJoCo disabled. Recording these values in the task configuration removes the need for training-script solver overrides.
The action parameters are also currently empty, so they inherit the default clip of
1.0. A policy output of2.0therefore becomes1.0before the task's offset-and-scale mapping. This also changes the action history used by observations and action-rate rewards. Explicitly settingclip: nullrestores the unbounded actions used by the successful historical training runs.The default changed during the action migration in #680. Explicit null support was added in #698, but the task YAMLs were left empty. Several task JSONs already declare
actions.joint_pos.clip: null; the Velocity environment does not forward that field. The issue is present onmainand was identified while validating #829.env.actions.joint_position.params.clip: nullin bothenv.yamlandenv.newton.yamlfor Go1, Go2, G1, H1_2, ANYmal-C and MicroDuck: 12 configurations.env.newton.yamlfiles tosolver_type: dexuni,joint_mode: dynamic,contact_mode: auto, and explicitmujoco_options:iterations: 20,ls_iterations: 50,use_mujoco_cpu: false,integrator: implicitfast,cone: pyramidal,impratio: 1.0,ls_parallel: false,separate_worlds: true,tolerance: 1e-8,njmax: 1024andnconmax: 256.newtonas the six task catalogs' default deployment and update their README training commands and solver description.ActionManager. Check joint ordering, unbounded action space, preserved raw actions and calibrated joint targets.Dependencies: none beyond the current
maindependencies.Type of change
Validation
On this main-based branch, 60 CPU tests pass. The regression checks fail with the original action clipping, solver selection and catalog defaults:
CUDA_VISIBLE_DEVICES='' PYTHONPATH=.:extensions:extensions/embodichain_tasks \ python -m pytest -q \ tests/gym/envs/tasks/test_unitree_locomotion.py \ tests/gym/envs/managers/test_default_joint_position.py \ tests/gym/envs/tasks/test_locomotion_deployments.py::test_packaged_deployment_and_agent_config_agree \ tests/test_locomotion_catalog.pyThe extended test closes the gap between generic action-term coverage and actual packaged task configuration. Existing action-history and explicit clipping tests are reused. These checks run on CPU without creating a simulation World.
On the current PR head
941c77366, all six Newton tasks also pass actual GPU training smoke runs: 4,096 environments, 24 steps/update, seed 42, 10 PPO updates from scratch (983,040 environment steps per task). The runs load the committed environment YAMLs directly; only the update limit is shortened and the environment file reference is resolved to its absolute path. The runtime is Python 3.11.15 with DexSim 0.6.0rc1 AOT.Each run verifies DexUni
pure_mujocodispatch, CUDA MuJoCo Warp with CPU MuJoCo disabled, solver iterations 20/50, unbounded actions and NoRender. All 60 PPO updates have finite losses and policy parameters. All six final resets return finite observations, and all six checkpoints save and reload successfully with the expected update/step counters and finite policy tensors.Previously completed controlled training used #829 at
c6e991c2e, whose Velocity binding and action manager match this branch's base460939c83. Each of the four Unitree tasks completed a separate 10-update startup check and 1,000 updates from scratch using DexUni → CUDA MuJoCo Warp, solver iterations 20/50, 4,096 environments, 24 steps/update and seed 42. The configuration change under comparison was onlyclip: null. The four updated Unitree environment YAMLs match those recorded null-clip training inputs exactly after YAML decoding. These 1k runs are reused as convergence evidence alongside the current-head GPU smoke runs above.Episode returns below are medians over updates 951–1,000. The nine-command evaluation uses 128 environments per command, 1,000 control steps and excludes the first 50 steps from error metrics.
Historical runs resumed at updates 8 and 30; the new runs trained continuously from zero. H1_2 matches the historical 1k pass count and still fails seven commands at that stage. ANYmal-C and MicroDuck pass the current-head 10-update GPU smoke runs; their longer-term training convergence remains unvalidated in this change.
black==26.10.1formatting and its repository-wide check pass. The public API documentation gate passes (4,434/4,434 exports). Task READMEs describe the new default. Theenv-frameworkandrl-learningcontext was reviewed: it already directs readers to the task-owned configurations, and its loading guidance remains accurate.Checklist
black .command to format the code base.python docs/scripts/check_api_docs.py), if applicable. No public API changes; coverage check passed.