Repository navigation
Add online world-model learning and APG motion tracking - #924
Conversation
han-xudong
left a comment
There was a problem hiding this comment.
Thanks for putting together the motion-tracking workflow and the supporting simulation and visualization APIs. I reviewed the implementation and checked the declared DexSim 0.5.1 dependency at the artifact level.
There are correctness issues to address before merging, particularly heading/next-frame alignment, episode provenance, marker texture ownership and sensor isolation, external checkpoint loading, and native-test setup. The inline comments include concrete reproductions and suggested corrections. I am requesting changes for those issues; the two P3 notes are non-blocking.
The PR-linked 0.5.1 artifact corresponds to source e3ecde7f0; the current tested 0.5.1 CI artifact corresponds to a8cf41e2b0. They are different builds despite sharing the 0.5.1 release version. The package index also serves a 0.5.1 artifact from 28970b7d0 that lacks interfaces required by this PR; that is an artifact-selection mismatch, not a PR implementation finding. Please identify the supported artifact by build identity rather than the release version alone.
One observed failure is specifically caused by using the newer 0.5.1 build: the original contact reader works on the PR-linked e3ecde7f0, but a8cf41e2b0 changes fetch() semantics and removes fetch_async(). This is a compatibility/migration item if the newer artifact is supported, not a defect against the PR's linked SDK. In contrast, replicated-articulation removal fails on both compared builds and in an SDK-only scene; that binding issue needs an SDK fix and is not explained by switching between these builds. These dependency issues are separate from the 13 PR findings.
Validation totals below use PR head 80297a877ae0e4404f9e9f8e231d13e5b58426f4 with tested CI build a8cf41e2b0; they are not a claim that the entire suite ran on both SDK builds. The original PR and local candidate each pass 1,624 selected CPU tests. The focused 25 reproductions give 20 failures/5 passes on the original PR and 25 passes on the candidate. The candidate passes 31/33 additional native/GPU/slow cases, including the corrected G1 tests. These candidate edits are not in the reviewed PR head. The remaining failures are opaque-ghost sensor visibility and SDK articulation removal. Python/data/example checks use simulation-boundary doubles where noted. The ghost repair is not implemented or validated; the failure-path SIGSEGV remains unresolved, and full-scale/long-run training and multi-seed convergence were not qualified locally.
| else {} | ||
| ), | ||
| ), | ||
| render_mode="scene" if scene_lighting else "overlay", |
There was a problem hiding this comment.
[P2] Keep opaque robot ghosts out of default sensor images
An opaque RobotMarkers instance with the default unlit=False takes this scene branch. It therefore enters the ordinary ray-traced scene, which the camera's visual-overlay opt-in does not control. With the tested 0.5.1 CI build a8cf41e2b0, the existing textured-ghost test changes the default camera from 0 to 3,990 red pixels before set_visual_overlay_enabled(True) is called. This violates the promised default sensor exclusion and can contaminate observations or recordings. Please resolve lighting and camera visibility as independent contracts, coordinating an SDK visibility channel/mask if needed. Forcing every marker into the existing overlay pass would remove the promised scene lighting and is not a complete fix. A concrete cross-SDK plan is to add a visualization visibility channel/mask independent of PBR routing, mark opaque ghosts with it, and have default cameras exclude it while opted-in cameras include it. Verify default exclusion, opted-in visibility, preserved lighting, and opacity transitions. This capability is not currently exposed; this is an unimplemented, unvalidated remediation plan, not a one-line fix. The pixel reproduction ran on the tested 0.5.1 a8 build. Source comparison shows the render-lowering function, native overlay policy, and shader are unchanged from the PR-linked 0.5.1 e3 build; the cited routing mechanism is not a change between those builds. A live pixel comparison on e3 was not performed.
There was a problem hiding this comment.
Confirmed; this remains open in #833, which owns RobotMarkers and native visualization. Opaque scene-lit ghosts currently enter the ordinary scene, while the camera opt-in controls the overlay pass. The correction will separate camera visibility from material lighting. Acceptance requires default-camera exclusion, opted-in visibility, preserved scene lighting and opacity transitions.
There was a problem hiding this comment.
Confirmed that this remains unresolved. I am keeping this existing thread open on #924 while the implementation is tracked in #833.
The acceptance criteria remain: exclusion from default sensor images, visibility after explicit opt-in, preserved scene lighting, and correct behavior across opacity transitions. Please provide native image-based validation once that correction is included in the prerequisite and synchronized into the complete workflow.
There was a problem hiding this comment.
The sensor-isolation correction is implemented in #833, with native image validation repeated at fb23a4994 using the declared DexSim 0.6.0rc1 dependency.
Opaque scene-lit markers retain their material/scene-lighting path and receive a dedicated render category. Default mono and stereo cameras exclude that category; include_visual_overlays=True includes it. The camera mask also preserves marker exclusion when ground visibility changes.
The native texture/material/opacity image regression passes for Hybrid mono/stereo and FastRT mono. It checks default sensor exclusion, opted-in visibility, opaque override and asset-material restoration, zero/translucent opacity, and removal. The post-merge selection passed 3 Hybrid and 2 FastRT tests, including the ground-visibility interaction.
#833 and #836 are still open. #924 still needs synchronization after those prerequisites merge, followed by the required integrated state-injection/reset and saved-trajectory-replay checks. The implementation and prerequisite validation are complete; that final integration condition remains pending.
773bfd0 to
2e7c41e
Compare
|
Twelve of the thirteen inline findings are implemented: four in this PR ( The installation instructions explicitly select the supplied runtime wheel, and the maintainer validation record identifies the tested build. Contact-reader migration for a newer runtime remains pending in the shared simulation integration owned by #836. Replicated-articulation removal remains a runtime lifecycle issue requiring its own fix and validation. Both #836 and #833 have been rebased onto main |
|
Thanks for addressing the feedback and adding focused regression coverage. I verified the corrections supporting the twelve resolved findings; those threads can remain resolved. Please address the two new inline findings: quaternion normalization before SLERP and the installation recipe’s Torch-profile conflict. The existing ghost sensor-isolation thread remains open. All eighteen marker tests pass with DexSim 0.5.1 build a8cf41e2b0. Their CI setup errors came from using 0.5.1rc1. Please identify the supported SDK artifact and align CI accordingly. If a8 is selected, the contact reader needs the corresponding API adaptation. Articulation removal remains a separate SDK issue. I am keeping “changes requested” pending these fixes and validation of the synchronized prerequisites. Please retain the historical-quality labels until corrected-heading evaluation is available. |
|
Regarding the follow-up review: The two new #924 findings are corrected. The existing ghost sensor-isolation finding remains with #833: default camera exclusion, explicit opt-in, scene lighting and opacity transitions still require native image validation. The twelve previously resolved findings retain their status. The runtime validation record identifies the runtime wheel and its source revision. CI must use that same artifact when the #836 prerequisite is integrated; the existing |
han-xudong
left a comment
There was a problem hiding this comment.
LGTM for the code changes in #924 at 34a05bc. The two latest findings are fixed and verified, with no remaining blocking code findings in the reviewed scope. Thanks for addressing the feedback.
Please merge only after #836 and #833 are completed and integrated, and integration CI passes. The existing ghost sensor-isolation thread remains open for dependency tracking.
[High impact] The PR adds behavior across learning, task planning, simulation, and data-collection workflows. The PR still needs the previously reported selected-row planning fix before merging.
|
yuecideng
left a comment
There was a problem hiding this comment.
Architecture review: naming, extension boundaries, and ODS ownership
Architecture follow-up for PR #924 at 34a05bc42aef31ded7efdb71360e1b26e15db3b9.
The overall layering is sound: differentiable environment contracts, RL updates, supervised model fitting, and online update scheduling have distinct responsibilities. Please make the scoped adjustments below in this PR. Broader abstractions should follow a second concrete use case. This comment records architectural requests, not new correctness findings or a revalidation of previously reported defects.
Requested in this PR
1. Use environment names that treat batching as the default contract.
EmbodiedEnv already supports concurrent environments. Keep batching explicit in the documented shapes and num_envs contract, without encoding it again as Vec in these protocol names. Use Protocol to distinguish structural interfaces from implementation base classes:
| Current name | Requested name |
|---|---|
LearningVecEnv |
LearningEnvProtocol |
DifferentiableVecEnv |
DifferentiableEnvProtocol |
ScheduledDifferentiableVecEnv |
ScheduledDifferentiableEnvProtocol |
EmbodiedEnv |
Keep unchanged |
Simulation base class DifferentiableEnv |
Keep unchanged |
Keep DifferentiableEnv as the unified, backend-neutral implementation entry point. Do not rename it to NewtonKinematicEnv. Backend-specific behavior belongs behind configuration/adapters/internal implementations. Document and validate the capabilities actually supported today; a unified name does not imply that every solver already supports differentiable dynamics.
Preserve compatibility aliases for existing public *VecEnv imports, and update internal references, public exports, type annotations, API documentation, examples, and project context together. Verify that existing downstream imports remain usable, including NMG's shared rollout interfaces.
2. Make the new scheduler's name and location match its scope.
- Move
embodichain/learning/dynamics/online.pytoembodichain/learning/dynamics/schedule.py. - Rename
OnlineLearningScheduletoOnlineModelBasedSchedule. - Rename
OnlineLearningCfgtoOnlineModelBasedScheduleCfg. - Keep the implementation under
learning/dynamics/; do not promote it tolearning/online.pyor introduce a general workflow package in this PR.
The component schedules physical collection, model-only warmup, model updates, and policy updates. It owns update credit and progress counters, while callbacks own data, optimizers, and completed backward work. Update imports, exports, tests, and documentation consistently.
3. State the supported extension boundary and validate recipe assumptions.
Keep MotionTraining explicitly documented as the G1/SuperMimic composition entry point. The reusable pieces are the environment protocols/registries, DifferentiableTrainer.update_once(), SupervisedLearner, and the callback-based schedule.
The current entry constructs KinematicWorldModel, selects a particular objective/replay schema, derives task-specific policy dimensions, and expects an actor-critic algorithm with a separate critic optimizer. Document these requirements and reject unsupported selections with a clear error at the assembly boundary. The presence of trainer / policy / algorithm blocks should not imply that changing a name in YAML is sufficient to run arbitrary algorithms or model families through this example.
4. Document the ODS boundary without merging storage implementations.
| Component | Ownership |
|---|---|
ODS OnlineDataEngine and data sources |
Data production, shared storage, sample selection, validity, and producer lifecycle |
ODS ODSFlowController |
Refill, bounded waits, delivery limits, and data-supply pacing |
OnlineModelBasedSchedule |
Ordering and ratio of collection/model/policy updates, warmup, and learning progress |
TransitionReplay |
Device-resident measured transitions, episode-local windows/history, and imagination starts |
These are complementary responsibilities. Keep the device-resident replay for the current synchronous training path. Specify ownership, episode boundaries, history padding, action semantics, control period, and validity masks. If ODS is later connected, its supply pacing must not independently duplicate the learning workflow's collection trigger or update accounting. ODS should not own actor/critic optimization or autograd lifecycle.
Follow-up work, not required in this PR
- Model/provider factories: When a second model is integrated, extract construction of the model, objective, replay adapter, and imagination-start provider from
MotionTraining. Avoid adding task/model conditionals throughout the existing example. - Algorithm-owned optimization: When supporting multiple algorithms in the online workflow, remove its fixed assumption of three optimizers. Put optimizer/scheduler ownership in the corresponding learner or an explicit component interface.
- Stateful and alternative models: Define hidden-state initialization, history burn-in, reset/detach, sampling, and checkpoint semantics when implementing an actual RNN, latent-state, or ensemble provider. These are not currently plug-and-play YAML substitutions.
- ODS integration: Add an explicit adapter through the existing
BatchSource/SequenceBatchSource/SupervisedBatchcontracts for an ODS-fed model-learning use case. Align sequence semantics before deciding what sampling code to share; do not force GPU replay into CPU shared-memory/locking machinery. - General workflow infrastructure: Consider
learning/workflows/only after multiple use cases demonstrate shared composition, evaluation, checkpoint, and recovery needs. - Unified environment capabilities: Extend backend/provider support behind
DifferentiableEnvincrementally, with capability and gradient tests. A universal backend implementation is not a prerequisite for this PR. - NMG migration: Any migration of NMG's own training loop to the shared trainer should be separate and preserve its complete K-dependent rollouts, objective scaling, effective-interaction counting, semantic normalization, and recovery behavior. The previously identified Franka preset/TCP upgrade issue is separate from this PR's APG architecture changes.
Scope and validation
World-model naming is explicitly unchanged: keep KinematicWorldModel and the existing model-family names in this PR.
For this cleanup, use focused import/alias, configuration, scheduler, and existing APG/trainer regression tests, plus affected API/context checks. The requested naming and boundary clarification does not itself require a new full-scale convergence run. This comment does not claim new runtime validation of the current head or supersede separate correctness and acceptance reviews.
|
@yuecideng The four changes requested in your architecture review are implemented in 1. Environment protocols and compatibilityThe canonical names are now
The protocol test checks alias identity at both public import paths. Shared rollout and trainer regressions pass for complete scheduled horizons, objective scaling, gradient accumulation, and checkpoint recovery. 2. Model-based scheduleThe implementation is now The schedule owns collection/model/policy ordering, model-only warmup, fractional update credit, and progress counters. Callbacks own their tensors, optimizers, and completed backward passes. Checkpoint fields and update calculations are unchanged; the existing prefill, warmup, callback-ordering, and resumed fractional-credit test passes. 3. G1 composition and recipe validation
The README documents the fixed 4. ODS and replay responsibilitiesThe README now defines ownership for The synchronous path uses device-resident replay. Future ODS integration will adapt Validation
The Newton planar-reach checks exposed a scalar-gravity incompatibility with the installed Newton API. Commit |
Retain upstream trajectory exports and quaternion regressions alongside floating-root kinematics and motion tracking.
| limits = authored_joint_velocity_limits( | ||
| self.robot, options.control_part | ||
| ).to(raw_result.positions) | ||
| success = torch.as_tensor( | ||
| raw_result.success, device=raw_result.positions.device, dtype=torch.bool | ||
| ) | ||
| valid = joint_velocity_validity(raw_result.positions, raw_result.dt, limits) |
There was a problem hiding this comment.
fixed_grid planning ignores the generator's selected rows when reading speed limits. For a robot with four environments and env_ids=(1,), the path has one row but authored_joint_velocity_limits() returns four. joint_velocity_validity() then raises instead of returning a plan. Selecting all rows in a different order can instead check the wrong limits.
Apply the same row selection to these limits. Also pass it to the new sequential compute_ik() and Cartesian-start compute_fk() calls, which currently default to all environments.
Prompt To Fix With AI
This is a comment left during a code review.
Path: embodichain/lab/sim/motion/motion_generator.py
Line: 815-821
Comment:
**Selected-row motion fails**
`fixed_grid` planning ignores the generator's selected rows when reading speed limits. For a robot with four environments and `env_ids=(1,)`, the path has one row but `authored_joint_velocity_limits()` returns four. `joint_velocity_validity()` then raises instead of returning a plan. Selecting all rows in a different order can instead check the wrong limits.
Apply the same row selection to these limits. Also pass it to the new sequential `compute_ik()` and Cartesian-start `compute_fk()` calls, which currently default to all environments.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Fixed in 9bbf45201. Fixed-grid velocity checks now select limits in env_ids order, and sequential IK and Cartesian-start FK receive the same environment selection. Continuous IK recovery also selects the corresponding joint bounds and velocity limits.
The regression cases cover a single selected row in a four-environment robot, all rows reordered as (3, 1, 0, 2), distinct per-row speed limits and base poses, and selected-row continuous recovery. These cases fail with the original implementation and pass after the correction. The targeted motion/configuration/APG suite passes all 189 CPU tests.
yuecideng
left a comment
There was a problem hiding this comment.
Follow-up review at 9bbf452 (base main@460939c83b6fc62f113b62af4798a89c42b3bec2).
The architecture cleanup is in place: canonical *EnvProtocol names retain exact compatibility aliases, DifferentiableEnv keeps its unified name, OnlineModelBasedSchedule lives in learning/dynamics/schedule.py, recipe assumptions fail early, and ODS/replay/learning ownership is documented. The selected-row planning fixes also look consistent on static review.
Three P2 issues remain in the inline comments: zero-weight critic minibatches still execute Adam updates; SDK installation instructions conflict with the pinned dependency; and resume compatibility does not identify changed motion contents.
Conditions for an initial infrastructure/experimental-workflow merge:
- Address these three P2 issues and pass focused regressions for the critic boundary and motion-content compatibility, plus verification of the corrected installation instructions.
- Merge prerequisites #836 and #833 into main, synchronize this branch, and pass required integration CI on the resulting commit using the declared DexSim 0.6.0rc1 dependency. Confirm physical state injection/reset and saved-trajectory replay against the integrated code; a conflict-free merge alone is insufficient.
- Preserve the distinction between initial infrastructure delivery and formal learning acceptance. Corrected-heading quality evaluation is pending; historical 0.94/full-clip figures must remain labeled historical and cannot establish current-version acceptance. Track corrected evaluation and the agreed training-budget evidence as follow-up acceptance work if the initial merge is scoped accordingly.
Model/provider factories, stateful-model interfaces, a general workflow package, and NMG trainer migration can remain follow-up work.
Validation from this review: 57 learning/model tests, 96 numerical tests, and 8 configuration/protocol/schedule tests passed; API coverage was 4489/4489 and the project-context check passed. The critic behavior and motion-identity gap were reproduced on CPU. Some native-dependent tests could not collect with the reviewer's locally installed older SDK, so this review does not claim current-head native/GPU integration or full-training qualification.
This COMMENT review records the outstanding work and merge conditions.
| if bool(torch.isfinite(loss) & torch.isfinite(norm)): | ||
| self.critic_optimizer.step() |
There was a problem hiding this comment.
[P2] Skip critic minibatches with zero total valid weight
Complete rollouts can contain a zero-weight suffix after early termination or invalid task data. Those rows are still shuffled into minibatches. A batch containing only zero weights produces zero loss and gradients, passes this finite check, and calls Adam.step(); existing momentum then changes the critic despite there being no valid samples.
Reproduced at this head with the existing LinearEnv, termination at step one, horizon three, and critic_minibatches=3: weights are [1, 0, 0]. With the valid row first, the two zero-gradient batches change the parameter by approximately -0.000670058 and -0.000517957, and critic_updates reports 3. Existing termination tests use one critic minibatch and miss this case. Please filter positive-weight rows before partitioning or skip batches whose total weight is zero, with a focused early-termination/multiple-minibatch regression.
There was a problem hiding this comment.
Implemented in a0058614f.
Fixed critic batching to select positive-weight samples before shuffling and partitioning. An empty valid set performs no critic optimizer step, so existing Adam momentum cannot change the parameters. Positive soft-survival weights retain their values, and critic_updates counts actual steps. Regressions compare the critic parameters and Adam state for a single valid transition versus the same transition followed by terminated/invalid suffixes, and verify that an all-invalid update preserves an already-initialized Adam state.
| The Python 3.11 dependency profile uses PyTorch 2.7, TorchVision 0.22 and | ||
| TorchCodec 0.5. The first install command selects CUDA 12.8 wheels for Torch and | ||
| TorchVision; the second installs TorchCodec's CPU decoder from PyPI. The SDK | ||
| requirement from #836 is `dexsim_engine>=0.5.1,<0.6`; use the review build selected above. |
There was a problem hiding this comment.
[P2] Align SDK installation instructions with the pinned dependency
This paragraph directs readers to select a review wheel in the range dexsim_engine>=0.5.1,<0.6, but the current pyproject.toml pins dexsim_engine==0.6.0rc1. The pinned release candidate is excluded by the documented range. A user following the stated range and supplying a 0.5.x wheel to the explicit wheel-install command above encounters a dependency conflict.
Please update the requirement and wheel-selection instructions to the published 0.6.0rc1 artifact used by current validation, and check the documented installation against that profile. This is separate from the earlier Torch-profile issue.
There was a problem hiding this comment.
Implemented in a0058614f.
Updated the install recipe to use the published dexsim_engine==0.6.0rc1 wheel from the SDK index, matching pyproject.toml, and removed the review-wheel and <0.6 instructions. I ran the revised commands in a new Python 3.11.15 environment; they resolved Torch 2.7.1+cu128, TorchVision 0.22.1+cu128 and TorchCodec 0.5. uv pip check, runtime imports and the installed embodichain run-example learning/motion_tracking/train.py --help entry all passed.
| "motion": { | ||
| "names": self.env.motion.names, | ||
| "lengths": self.env.motion.lengths.cpu().tolist(), | ||
| "fps": self.env.motion.fps, | ||
| }, |
There was a problem hiding this comment.
[P2] Include motion contents in checkpoint compatibility identity
If motion.npz is regenerated or replaced at the same motion_path with identical clip names, lengths, FPS, and joint order but different qpos/qvel values, this identity remains unchanged. restore() then accepts the old policy/critic, normalization statistics, and adaptive sampling history for a different reference dataset. MotionLibrary and sampler validation do not add a content check downstream.
A CPU probe at this head using otherwise identical libraries with joint trajectories differing by 0.5 rad produces equal training motion identities, and the sampler accepts the old state. Please persist a stable digest of the selected motion contents and compare it before restoring any learning state. A same-layout/different-content rejection case can cover this without simulation. URDF compatibility already has a separate model extra-state check.
There was a problem hiding this comment.
Implemented in a0058614f.
Added a versioned content identity to MotionLibrary, computed once from the selected float32 motion data, clip layout, joint order and sampling frequency before device upload. Both the training checkpoint and sampler state retain it, and restore checks it before loading learning state. Regressions accept unchanged data, reject qpos/qvel changes at the same path with unchanged clip metadata, ignore changes to unselected clips, and verify rejection leaves state untouched. Checkpoints without the content identity are rejected explicitly; this compatibility change is documented.
Merge main through #924 while preserving named and direct provider configuration and capability-aware articulation reset. Validate NeRD with the shared collector/trainer, including pre-reset terminal gradients and a real APG Actor-Critic update. Repair existing PR CI failures in test fixtures, visualization declarations and runtime-version comparison.
Description
This PR adds online world-model learning and APG Actor-Critic with a G1 motion-tracking task. A physical environment collects state transitions, supervised learning updates the world model, and the policy learns by backpropagating through short model rollouts. Physical collection, the model task environment, and independent evaluation share observation, reward, and termination calculations.
Refs #848
Changes
APGActorCriticfor short-rollout policy updates and value regression with lambda-return targets. Policy updates freeze world-model and critic parameters while retaining state and action gradients; normalization statistics remain fixed during rollouts.OnlineModelBasedSchedulefor collection, model warmup, and model/policy update ordering and ratios; add replay history and task-progress fields, and multi-step dynamics training through the sharedSupervisedLearner.trainer / policy / algorithm:MotionTrackingModelRL,motion_tracking, andapg_actor_critic, with a fixed kinematic world model and replay schema. The online workflow selects collection and evaluation environments separately and injects the shared WM, motion library, and replay start provider. Checkpoint recovery preserves optimizers, scheduling, and sampling state.Usage and dependencies
Entry points in
examples/learning/motion_tracking/run throughembodichain run-examplefor data preparation, model download, training, evaluation, and playback. Data preparation uses a pinned upstream revision and preserves attribution to Ubisoft, the Unitree retargeting project, and the download mirror. Pretrained weights are available in embodichain_model. The example README provides complete commands.The physical task uses Newton/MJWarp and depends on the shared simulation interfaces in #836. FastRT saved-trajectory playback uses RobotMarkers from #833.
Review order
This branch contains the learning and G1 task changes on
main. The physical task uses the simulation APIs in #836, and saved-trajectory playback uses RobotMarkers from #833. Run integration CI after both prerequisites merge intomain.examples/learning/motion_tracking/README.md,g1_motion/agents/model_based.yamllearning/rl/algo/apg_actor_critic.py, differentiable collector and trainerlearning/dynamics/schedule.py,embodichain_models/dynamics/rwm/embodichain_tasks/locomotion/tracking/, motion policy,compute/kinematics/examples/learning/motion_tracking/Validation with DexSim 0.6.0rc1
The branch uses the published wheel pinned by #1124. An earlier GitHub test run found a missing explicit visualization configuration in
replay.py. The headless exporter now declares its live visualization backend as disabled. The failing example-coverage check passes locally. Previous CI failureembodichain run-exampleentry exported and decoded five FastRT frames at 320 × 240 and 50 FPS with the reference robot visible and zero physics steps.Torch 2.7 full-graph compilation
The Python 3.11 CI lane uses Torch 2.7.0+cu128. A default-device context made Dynamo fail on
Tensor.unflattenduring heading extraction. The same condition also affected compiled model transitions and supervised rollout loss in local reproduction. These paths now use equivalent reshape operations. The existing regressions explicitly exercise a device context and check values and gradients. Failing CI runDiff against the current target commit · GitHub checks
Earlier review and validation
Commit
fe744372caddresses the architecture review: canonical*EnvProtocolnames with exact*VecEnvcompatibility aliases,OnlineModelBasedScheduleinlearning/dynamics/schedule.py, early G1 recipe validation, and documented ODS/schedule/replay ownership.DifferentiableEnvand world-model family names retain their existing names.b5961bdb9.Commit
58a02e62bpreserves explicit options in a separate evaluation configuration. Collection settings fill omitted fields, except clip selection; omitted or nullclipsevaluates all clips in the selected archive. When both stages resolve to the same configuration file, evaluation retains workflow and CLI overrides. The README documents these precedence rules.Commit
34a05bc42normalizes quaternion endpoints before SLERP and aligns the Python 3.11 installation recipe with the declared Torch profile. Earlier commit2e7c41ef6corrects next-frame reference control, independent heading alignment, restricted external WM loading, and the G1 solver configuration override. The branch is based onmain@c1f90ce7b.2e7c41ef6, 164 focused CPU tests passed across kinematics, motion data, task calculations, APG, trainer configuration, scheduling, replay and dynamics models; six native/GPU cases were deselected.The figures below use the previous heading calculation. The correction changes body-pose reward and evaluation metrics; quality measurements with the corrected calculation are pending.
Historical training and integration validation
G1/LAFAN1 training covers 40 motion clips with 4,096 parallel environments and 50 Hz control. Each seed completes 60,000 vector collection steps, 240,000 WM updates, 24,000 actor updates, and 480,000 critic updates.
Installed FastRT playback passes: 4 seconds at 1080p/50 fps, displaying the simulated robot, green reference robot, red/green tracking points, and clip timeline, with zero physics steps during playback.
Type of change
Screenshots
Historical training and evaluation figures
Left: online WM training loss, showing raw values and an EMA (α=0.08) against vector collection iterations. Right: independent evaluation on the 40 training clips; each dot represents one clip and each diamond is the equally weighted clip mean. Each seed uses 4,096 environments and 60,000 collection iterations on an RTX PRO 5000 72 GB.
The frozen published model predicts 1/8/16/32 steps from 128 complete windows of newly collected physical trajectories using measured actions. The figure compares the WM with holding the current state. Control runs at 50 Hz; model parameters and normalization statistics remain fixed. At 32 steps, mean root position error is 2.92 cm and joint position RMSE is 0.0291 rad.
Continuous multi-motion policy evaluation
g1-continuous-180s.mp4
The seed-0 policy tracks 16 reference segments from 11 source clips, completing 15 transitions over 180 seconds with zero resets. Mean world-space root XY error is 9.36 cm. After root heading and horizontal-position alignment, the mean per-frame tracked-body position RMS error is 3.59 cm. The evaluation was recorded on 2026-10-31 and rendered from saved states on 2026-11-02 at 1920×1080 and 50 fps, with a 300×350-pixel World XY panel.
Checklist
black .command to format the code base.python docs/scripts/check_api_docs.py), if applicable.