Skip to content

Let PickUp choose the most reachable outward tilt of its approach - #1246

Open
Yuan-Xinyi wants to merge 1 commit into
mainfrom
feat/pickup-approach-tilts
Open

Yuan-Xinyi wants to merge 1 commit into
mainfrom
feat/pickup-approach-tilts

Conversation

@Yuan-Xinyi

@Yuan-Xinyi Yuan-Xinyi commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Description

This PR lets PickUp consider outward tilts of its approach direction and use the one that leaves the most reachable grasp candidates. It is opt-in: with the new option unset, PickUp behaves as before.

Related: #1247 (same file, independent change; see "Relation to #1247").

Problem and cause

PickUp samples grasps for one world approach direction, vertical by default (pick_up.py#L183). Every candidate must have an IK solution for the pre-grasp pose 15 cm back along that direction, the grasp, the lift and the downstream targets (#L961-L1055). When none has, planning fails; no other direction is tried.

A short arm reaches little with a vertical gripper. With a dual ARX5 (X5A) on RoboTwin's tabletop layout, GenSim deployments failed in motion planning in two ways.

No candidate at all. Bottles standing 57 to 66 cm from the arm base:

PickUp found no candidate with a feasible vertical pickup path: aligned=[200], pre_grasp=[0], grasp=[0], lift=[0], downstream=[[0]].

One to five candidates out of 100. The cube that has to be carried across the table midline. The pick is planned from whichever candidate is left, and the placement that follows then fails in the planner:

Pick Reachable candidates Outcome
blocks_ranking_rgb, third cube (plan A) 0 pick not planned
blocks_ranking_rgb, third cube (plan B) 1 placement not planned
blocks_ranking_size, third cube 5 placement not planned

The figure shows where a downward grasp and its pre-grasp have an IK solution, for the default direction and for two outward tilts. It is a lower bound: IK is solved from the zero seed, and the solver converges only from nearby seeds.

Reach of both arms for three approach directions

Changes

  • PickUpOptions.approach_tilts (pick_up.py), a tuple of angles in radians, empty by default. For each angle the approach direction is rotated outward, toward the horizontal direction from the arm base to the object, and grasp sampling and the reachability screen are repeated. Each row uses the direction that leaves the most reachable candidates; approach_direction wins ties, then the smaller angle. Pre-grasp and trajectory use the chosen direction; the lift stays along world z.
  • The sample metadata records the chosen angle per row as approach_tilts.
  • The configured decoder accepts approach_tilts for pick_up (integrations/configured.py).

Explicit goal grasps and fixed_object_to_eef bypass sampling and are not affected.

Results

Seven generated deployments that failed in motion planning, re-executed with the same plans and scenes. Dual ARX5, cuRobo, default skill options except approach_tilts: [0.2618, 0.4363, 0.6109] (15, 25 and 35 degrees) on every pick. The checkout is main plus the open GenSim PRs (#1225, #1226, #1231, #1235, #1236, #1237, #1238, #1243) and a local ARX5 gripper description. Task success, first attempt of each run:

Deployment Neither option approach_tilts (this PR) #1247 only Both
pick_diverse_bottles 0/1 1/1 0/1 1/1
pick_dual_bottles 0/1 1/1 0/1 1/1
blocks_ranking_rgb, plan A 0/3 2/3 0/3 3/3
blocks_ranking_rgb, plan B 0/3 3/3 0/3 3/3
blocks_ranking_size 0/3 2/3 3/3 2/3
Total 0/11 9/11 3/11 10/11

Reachable candidates of the picks listed above, without and with the option:

Pick Default direction With approach_tilts Chosen tilt
blocks_ranking_rgb, third cube (plan A) 0 64 35°
blocks_ranking_rgb, third cube (plan B) 1 75 35°
blocks_ranking_size, third cube 5 89 35°
pick_diverse_bottles, orange bottle 0 9 35°
pick_dual_bottles, cola bottle 0 82 35°

The two failures in the approach_tilts column: one placement of the third cube was not planned (blocks_ranking_rgb, plan A), and one placement failed its effect verification after release (blocks_ranking_size). The failure in the last column is a neighbouring cube pushed 2 cm by the opening fingers.

Two more deployments in the set still fail, for reasons outside this change: handover_block (the pick is now planned, but the arm scrapes the table on its way and knocks the block, because the planner world has no table) and handover_mic (the hand-over transfer is not planned).

Limitations

  • The choice is by candidate count, so easy picks change too: a cube 37 cm from the arm base had 94 reachable candidates along the default direction and 100 with a 25° tilt, and was picked with the tilt. With the largest tilt the gripper ends up 40 to 53 degrees from vertical on the far cubes. No grasp failed because of that in these runs, but the option does change how objects are held.
  • Each angle repeats grasp sampling and four or more batched IK solves, about 1.5 to 2 s per angle here.
  • Rows directly above or below the arm base have no outward direction and keep the configured approach.
  • The reach figure and the counts come from one robot and one table layout.

Relation to #1247

Both PRs add one field to PickUpOptions and one key to the pick_up decoder, next to each other, so the second one to merge needs a small rebase. They are independent in behaviour; the table above shows each alone and both together.

Validation

  • pytest tests/sim/atomic_skills tests/lab/task_program tests/gym/envs/task_program -m "not gpu" on CPU: 3297 passed, 6 skipped.
  • New coverage: one test of the direction choice (a row that keeps more candidates along the configured direction, a row that gains them with the tilt; chosen direction, pose, metadata rows and recorded angle).
  • black --check ., python docs/scripts/check_api_docs.py and the agent-context check pass. agent_context/topics/atomic-actions/execution.md records the option.

Type of change

  • New feature (non-breaking change which adds functionality)

Screenshots

See the figure above.

Checklist

  • I have run the black . command to format the code base.
  • I reviewed affected documentation and agent context, updated it where needed, or explained why no update was needed.
  • Public API changes are reflected in the API docs (python docs/scripts/check_api_docs.py), if applicable
  • I verified appropriate behavioral coverage, reviewed the necessity of added or extended tests, and explained when no new tests were needed.
  • Dependencies have been updated, if applicable.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
@Yuan-Xinyi Yuan-Xinyi added atomic-skill Atomic Skill goals, planning, execution, verification and recovery. feature New user-facing capabilities. labels Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

atomic-skill Atomic Skill goals, planning, execution, verification and recovery. feature New user-facing capabilities.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant