Skip to content

Let a pickup screen the pose from which the next placement approaches - #1247

Open
Yuan-Xinyi wants to merge 1 commit into
mainfrom
fix/pick-screens-place-approach
Open

Yuan-Xinyi wants to merge 1 commit into
mainfrom
fix/pick-screens-place-approach

Conversation

@Yuan-Xinyi

@Yuan-Xinyi Yuan-Xinyi commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Description

This PR lets a pickup also check the pose from which the next placement approaches its target, and makes that check less dependent on the IK seed. It is opt-in: with the new option unset, pickup screening is unchanged.

Related: #1246 (same file, independent change; see "Relation to #1246").

Problem and cause

Before a grasp is chosen, PickUp checks that the arm can reach the downstream object targets with it (pick_up.py#L1024-L1046). For a following Place the compiler passes the placement target (lowering.py#L1953-L1972). Two things make this check differ from what Place then does.

1. Place does not go to the target directly. It first moves to the pose lift_height (10 cm) above it (place.py#L286). That pose is not screened, so a grasp can pass the pickup and leave the placement without a plan.

2. The check solves IK from one seed. Each downstream pose is solved from the solution of the pose before it, starting at the lift pose (pick_up.py#L1024). The numerical solver converges from a nearby seed and often not from a distant one, so near the edge of the workspace the result depends on the order:

  • blocks_ranking_rgb, third cube: no candidate reached the placement target from the lift pose (downstream=[[0]]). With the approach pose solved first and the target solved from it, candidates passed, and the placement was executed.
  • blocks_ranking_size, second cube: the existing check selects a grasp with which the cube is then placed. Screening the approach pose in that fixed order no longer selected it and took its mirrored variant instead, which missed the 82 mm cube with an 88 mm gripper in 4 of 4 runs.

Changes

  • PlaceOptions.approach_clearance (place.py): the world-z distance of the approach pose above the release pose, zero when max_approach_retract_z caps it.
  • SemanticObjectTarget.approach_clearance (compiler/lowering.py): recorded by the built-in Place look-ahead and by the relative-place lowerer (integrations/_configured_services.py).
  • PickUpOptions.screen_downstream_approach (pick_up.py), false by default. With it:
    • the compiler adds the raised object pose before each such target in downstream_object_target_poses; a late-bound scene target stays late-bound, the offset goes into its world displacement;
    • a downstream pose that has no IK solution from the pose before it is tried again from the lift pose and from the pose that follows it. A pose counts as reachable when any of these converges.
  • The configured decoder accepts screen_downstream_approach for pick_up (integrations/configured.py).

Results

Seven generated deployments that failed in motion planning, re-executed with the same plans and scenes. Dual ARX5, cuRobo, default skill options except the options named in the header. The checkout is main plus the open GenSim PRs (#1225, #1226, #1231, #1235, #1236, #1237, #1238, #1243), #1246 and a local ARX5 gripper description. Task success, first attempt of each run:

Deployment Neither option #1246 only screen_downstream_approach (this PR) Both
pick_diverse_bottles 0/1 1/1 0/1 1/1
pick_dual_bottles 0/1 1/1 0/1 1/1
blocks_ranking_rgb, plan A 0/3 2/3 0/3 3/3
blocks_ranking_rgb, plan B 0/3 3/3 0/3 3/3
blocks_ranking_size 0/3 2/3 3/3 2/3
Total 0/11 9/11 3/11 10/11

What this PR does on its own is limited:

Earlier revisions of this change, measured on the same block deployments together with a first-feasible version of #1246:

Revision blocks_ranking_rgb (two plans) blocks_ranking_size
approach pose screened, fixed seed order 8/8 0/4
approach pose screened, seed retries (this PR) 2/6 3/3

The fixed order rejected a workable grasp on blocks_ranking_size; the retries accept more marginal grasps on blocks_ranking_rgb. #1246 in its final form removes most of that marginal regime by choosing the approach with the most reachable candidates.

Limitations

  • The screen is still an IK check with a numerical solver. It remains an approximation of what the motion planner can do, with errors in both directions near the workspace edge.
  • A capped approach (max_approach_retract_z) is not screened, because its height depends on the realized release pose.
  • Only Place and the relative-place lowerer declare an approach clearance. Other registered calls keep their look-ahead targets as they are.
  • With the option, up to three IK solves replace one per downstream pose.

Relation to #1246

Both PRs add one field to PickUpOptions and one key to the pick_up decoder, next to each other, so the second one to merge needs a small rebase.

Validation

  • pytest tests/sim/atomic_skills tests/lab/task_program tests/gym/envs/task_program -m "not gpu" on CPU: 3301 passed, 6 skipped.
  • New coverage: the compiler adds the raised pose before a Place target only on request and not for a capped approach; the relative-place look-ahead records the clearance and the raised target stays late-bound; the screen retries from the lift pose and from the following pose with the option and keeps the single solve without it.
  • black --check ., python docs/scripts/check_api_docs.py and the agent-context check pass. agent_context/topics/task-programs/execution.md records the behaviour.

Type of change

  • Enhancement (non-breaking change which improves an existing functionality)

Screenshots

Not applicable.

Checklist

  • I have run the black . command to format the code base.
  • I reviewed affected documentation and agent context, updated it where needed, or explained why no update was needed.
  • Public API changes are reflected in the API docs (python docs/scripts/check_api_docs.py), if applicable
  • I verified appropriate behavioral coverage, reviewed the necessity of added or extended tests, and explained when no new tests were needed.
  • Dependencies have been updated, if applicable.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
@Yuan-Xinyi Yuan-Xinyi added enhancement Improvements to existing capabilities; use feature for new capabilities. atomic-skill Atomic Skill goals, planning, execution, verification and recovery. task-program Task Program language, semantics, compiler, runtime and integration contracts. labels Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

atomic-skill Atomic Skill goals, planning, execution, verification and recovery. enhancement Improvements to existing capabilities; use feature for new capabilities. task-program Task Program language, semantics, compiler, runtime and integration contracts.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant