Repository navigation
Let a pickup screen the pose from which the next placement approaches - #1247
Open
Yuan-Xinyi wants to merge 1 commit into
Open
Yuan-Xinyi wants to merge 1 commit into
Yuan-Xinyi wants to merge 1 commit into
Conversation
Co-Authored-By: Claude <noreply@anthropic.com>
4 of 5 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR lets a pickup also check the pose from which the next placement approaches its target, and makes that check less dependent on the IK seed. It is opt-in: with the new option unset, pickup screening is unchanged.
Related: #1246 (same file, independent change; see "Relation to #1246").
Problem and cause
Before a grasp is chosen, PickUp checks that the arm can reach the downstream object targets with it (
pick_up.py#L1024-L1046). For a following Place the compiler passes the placement target (lowering.py#L1953-L1972). Two things make this check differ from what Place then does.1. Place does not go to the target directly. It first moves to the pose
lift_height(10 cm) above it (place.py#L286). That pose is not screened, so a grasp can pass the pickup and leave the placement without a plan.2. The check solves IK from one seed. Each downstream pose is solved from the solution of the pose before it, starting at the lift pose (
pick_up.py#L1024). The numerical solver converges from a nearby seed and often not from a distant one, so near the edge of the workspace the result depends on the order:blocks_ranking_rgb, third cube: no candidate reached the placement target from the lift pose (downstream=[[0]]). With the approach pose solved first and the target solved from it, candidates passed, and the placement was executed.blocks_ranking_size, second cube: the existing check selects a grasp with which the cube is then placed. Screening the approach pose in that fixed order no longer selected it and took its mirrored variant instead, which missed the 82 mm cube with an 88 mm gripper in 4 of 4 runs.Changes
PlaceOptions.approach_clearance(place.py): the world-z distance of the approach pose above the release pose, zero whenmax_approach_retract_zcaps it.SemanticObjectTarget.approach_clearance(compiler/lowering.py): recorded by the built-in Place look-ahead and by the relative-place lowerer (integrations/_configured_services.py).PickUpOptions.screen_downstream_approach(pick_up.py), false by default. With it:downstream_object_target_poses; a late-bound scene target stays late-bound, the offset goes into its world displacement;screen_downstream_approachforpick_up(integrations/configured.py).Results
Seven generated deployments that failed in motion planning, re-executed with the same plans and scenes. Dual ARX5, cuRobo, default skill options except the options named in the header. The checkout is
mainplus the open GenSim PRs (#1225, #1226, #1231, #1235, #1236, #1237, #1238, #1243), #1246 and a local ARX5 gripper description. Task success, first attempt of each run:screen_downstream_approach(this PR)pick_diverse_bottlespick_dual_bottlesblocks_ranking_rgb, plan Ablocks_ranking_rgb, plan Bblocks_ranking_sizeWhat this PR does on its own is limited:
blocks_ranking_size(0/3 to 3/3): the third cube's placement is now planned.blocks_ranking_rgbon its own. The pickups pass with 4 or fewer of 100 candidates, and the placement planner then fails on them. The IK screen and the motion planner (cuRobo) disagree on such marginal poses in both directions.Earlier revisions of this change, measured on the same block deployments together with a first-feasible version of #1246:
blocks_ranking_rgb(two plans)blocks_ranking_sizeThe fixed order rejected a workable grasp on
blocks_ranking_size; the retries accept more marginal grasps onblocks_ranking_rgb. #1246 in its final form removes most of that marginal regime by choosing the approach with the most reachable candidates.Limitations
max_approach_retract_z) is not screened, because its height depends on the realized release pose.Relation to #1246
Both PRs add one field to
PickUpOptionsand one key to thepick_updecoder, next to each other, so the second one to merge needs a small rebase.Validation
pytest tests/sim/atomic_skills tests/lab/task_program tests/gym/envs/task_program -m "not gpu"on CPU: 3301 passed, 6 skipped.black --check .,python docs/scripts/check_api_docs.pyand the agent-context check pass.agent_context/topics/task-programs/execution.mdrecords the behaviour.Type of change
Screenshots
Not applicable.
Checklist
black .command to format the code base.python docs/scripts/check_api_docs.py), if applicable🤖 Generated with Claude Code