Repository navigation
Let PickUp choose the most reachable outward tilt of its approach - #1246
Open
Yuan-Xinyi wants to merge 1 commit into
Open
Yuan-Xinyi wants to merge 1 commit into
Yuan-Xinyi wants to merge 1 commit into
Conversation
Co-Authored-By: Claude <noreply@anthropic.com>
4 of 5 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR lets PickUp consider outward tilts of its approach direction and use the one that leaves the most reachable grasp candidates. It is opt-in: with the new option unset, PickUp behaves as before.
Related: #1247 (same file, independent change; see "Relation to #1247").
Problem and cause
PickUp samples grasps for one world approach direction, vertical by default (
pick_up.py#L183). Every candidate must have an IK solution for the pre-grasp pose 15 cm back along that direction, the grasp, the lift and the downstream targets (#L961-L1055). When none has, planning fails; no other direction is tried.A short arm reaches little with a vertical gripper. With a dual ARX5 (X5A) on RoboTwin's tabletop layout, GenSim deployments failed in motion planning in two ways.
No candidate at all. Bottles standing 57 to 66 cm from the arm base:
One to five candidates out of 100. The cube that has to be carried across the table midline. The pick is planned from whichever candidate is left, and the placement that follows then fails in the planner:
blocks_ranking_rgb, third cube (plan A)blocks_ranking_rgb, third cube (plan B)blocks_ranking_size, third cubeThe figure shows where a downward grasp and its pre-grasp have an IK solution, for the default direction and for two outward tilts. It is a lower bound: IK is solved from the zero seed, and the solver converges only from nearby seeds.
Changes
PickUpOptions.approach_tilts(pick_up.py), a tuple of angles in radians, empty by default. For each angle the approach direction is rotated outward, toward the horizontal direction from the arm base to the object, and grasp sampling and the reachability screen are repeated. Each row uses the direction that leaves the most reachable candidates;approach_directionwins ties, then the smaller angle. Pre-grasp and trajectory use the chosen direction; the lift stays along world z.approach_tilts.approach_tiltsforpick_up(integrations/configured.py).Explicit goal grasps and
fixed_object_to_eefbypass sampling and are not affected.Results
Seven generated deployments that failed in motion planning, re-executed with the same plans and scenes. Dual ARX5, cuRobo, default skill options except
approach_tilts: [0.2618, 0.4363, 0.6109](15, 25 and 35 degrees) on every pick. The checkout ismainplus the open GenSim PRs (#1225, #1226, #1231, #1235, #1236, #1237, #1238, #1243) and a local ARX5 gripper description. Task success, first attempt of each run:approach_tilts(this PR)pick_diverse_bottlespick_dual_bottlesblocks_ranking_rgb, plan Ablocks_ranking_rgb, plan Bblocks_ranking_sizeReachable candidates of the picks listed above, without and with the option:
approach_tiltsblocks_ranking_rgb, third cube (plan A)blocks_ranking_rgb, third cube (plan B)blocks_ranking_size, third cubepick_diverse_bottles, orange bottlepick_dual_bottles, cola bottleThe two failures in the
approach_tiltscolumn: one placement of the third cube was not planned (blocks_ranking_rgb, plan A), and one placement failed its effect verification after release (blocks_ranking_size). The failure in the last column is a neighbouring cube pushed 2 cm by the opening fingers.Two more deployments in the set still fail, for reasons outside this change:
handover_block(the pick is now planned, but the arm scrapes the table on its way and knocks the block, because the planner world has no table) andhandover_mic(the hand-over transfer is not planned).Limitations
Relation to #1247
Both PRs add one field to
PickUpOptionsand one key to thepick_updecoder, next to each other, so the second one to merge needs a small rebase. They are independent in behaviour; the table above shows each alone and both together.Validation
pytest tests/sim/atomic_skills tests/lab/task_program tests/gym/envs/task_program -m "not gpu"on CPU: 3297 passed, 6 skipped.black --check .,python docs/scripts/check_api_docs.pyand the agent-context check pass.agent_context/topics/atomic-actions/execution.mdrecords the option.Type of change
Screenshots
See the figure above.
Checklist
black .command to format the code base.python docs/scripts/check_api_docs.py), if applicable🤖 Generated with Claude Code