Repository navigation
feat(gym): align expert and policy substep EEF control - #985
Merged
Merged
Conversation
Signed-off-by: yuecideng <dengyueci@qq.com>
Signed-off-by: yuecideng <dengyueci@qq.com>
7 of 10 tasks
yuecideng
marked this pull request as ready for review
November 5, 2026 11:23
Signed-off-by: yuecideng <dengyueci@qq.com>
[Medium impact] The PR appears safe to merge; no blocking issue remains. SummaryAdds
All four earlier, unnumbered findings are addressed: arena/TCP coordinates, solver joint order, completed-row holds, and recording cadence. No new actionable issues were found in the changes since the previous review.
|
Signed-off-by: yuecideng <dengyueci@qq.com>
Signed-off-by: yuecideng <dengyueci@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Let Task Program experts and policies execute through one configured EEF controller. RLinf's existing Stack Cube checkpoints use unbounded relative requests and a target held across physics substeps; the existing bounded, single-step contract has different semantics.
eef_pose.relative_substep_dls@1(RelativeEefDlsAction). Reuse the existing solver-order binding and arena/TCP Jacobian adaptation, while keeping each action contract's bounds and timing separate.EmbodiedEnv, substep counting/refresh/masking inActionManager, and joint-command generation in action terms.BaseEnvandSimulationManagerhave no PR diff. The environment connectsActionManager.get_substep_callback(num_substeps)to the existing batchedupdate(after_substep=...)path. Each callback owns one interval, skips refresh after the final physics step and preserves recording/visualization publication.expert_trajectory.use_action_manager, default false. Position-only experts retain their exact FK target and existing storage schema. Completed rows keep measured joint holds after conversion and refresh, including partly closed binary grippers.open_on_zero, default true. Stack Cube selects false to preserve its zero-closes boundary.The PR is based on main
16c429bc48ba0bc09e2256be26d8994fcb3b5e0b. Compatibility was also checked in a temporary merge with main184fa6031fc8e377441c7e936962064b1466621a. No new dependencies or solver/physics backend changes. Consumer: yuecideng/RLinf#8. Related #836 and #959 address broader optimization and stability work.Review corrections and simplification
managers.__all__.Removed the single-use expert-conversion and validation helpers, eliminated repeated expert-target FK, and replaced a duplicated DLS test implementation with a closed-form expectation. The new controller class decreased from 195 to 175 lines. Shared private frame/order helpers follow the existing action binding rather than adding a controller framework. The manager exposes one interval callback in place of a separate refresh flag and immediate-refresh method; the environment contains no substep counter or action-refresh policy. The two stateful manager-reset fixes remain because they are required by episode ownership.
Frame adaptation preserves the solver Jacobian's strides for substep DLS; the bounded controller retains its original contiguous layout. This avoids changing CUDA reduction order while correcting the geometry.
Type of change
Validation
Python 3.11.14, Torch 2.7.1+cu128, NumPy 2, DexSim 0.5.1rc1:
black==26.10.1:black ., passed.tests/sim/test_sim_manager.py: 13 passed, including realSimulationManager.update/render scheduling against a fake world, physics-rate recording, final-frame camera sharing and substep callbacks.e9cfe070has 6 assertion failures / 4 passes; the fixed head passes all 10. Failures measure coordinates, joint order, completed-row commands and missing frames rather than missing mock methods.python docs/scripts/check_api_docs.py: 3733/3733 exports documented. Context checks passed. Sphinx dummy build completed with the same 839 existing warnings as the main baseline; normalized warning sets are identical.Physical RTX PRO 5000 72 GB, GPU 5:
Earlier independent RLinf PPO integration collected two 450-step rollouts, performed one update and saved DCP plus full weights: 198 Adam states at step 1, 477 frozen tensors unchanged and 675 finite exported tensors. That training integration was not rerun for this review; the new validation above checks the affected control/lifecycle contracts. No new learning-quality claim, full repository/backend/multi-node qualification or formal training run is included. Remote CI results remain separate from local validation. AI assistance was used for review, implementation and validation.
Screenshots
No UI change. Raw camera and recording cadence are covered above.
Checklist
black .command to format the code base.