Repository navigation
Conversation
5 tasks done
[Medium impact] The PR appears safe to merge; no actionable issues were found. SummaryAdds optional outcome-guided grasp retries and faster selected-link reads during CPU planning.
No actionable issues found.
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Eight-environment trajectory collection can spend most of its time finishing grasp candidates that fail late. This follow-up to #1244 uses the physical outcomes already available to the collector to schedule subsequent attempts.
trajectory.affordance.branch_selectiontooutcome_guidedto try every non-nominal branch, then use accepted trajectories and verified task segments to select further attempts. The downstream recipes reserve 25% of post-bootstrap assignments for exploration. Existing recipes continue to use round-robin selection by default.How one retry works
flowchart TD A["Try every non-nominal branch"] --> B["Choose the next branch from physical outcomes and exploration"] B --> C["Generate valid grasps in the current layout"] C --> D{"Does a current candidate match the selected successful grasp in object coordinates?"} D -->|Yes| E["Use the matched candidate in the new layout"] D -->|No or exploring| F["Use the current branch ranking"] E --> G["Plan, execute and validate the full attempt"] F --> G G --> BIn the measured Tianji run, the third attempt successfully grasped the marker. In the marker's own frame the grasp translation was approximately (+20.7, −36.3, +50.3) mm. Attempts five and six found the same relative position and orientation in new candidate sets. Both executed a newly planned trajectory and passed. For CPU link reads, the corresponding scale is 8 environments × 30 links in a full read and 3 active environments × 2 requested arm roots in the measured selected read.
Before and after
Each task ran on the same host, serially, with Python 3.12, DexSim 0.6.0, one World, eight CPU physics environments and identical seeds, attempt budgets and acceptance criteria. Both timing arms included the same pre-existing local grasp-scoring edit. For each arm, all three operators produced three accepted episodes each, and all nine were exported to LeRobot and read back. Collection includes World initialization. The measured headless workflow also includes scene construction, export, readback and artifact registration.
For Tianji, failed grasp attempts consumed 31,964 → 7,384 environment action steps. Its grasp operator took approximately 721.0 → 216.0 s. Two later accepted episodes matched an earlier successful object-relative grasp in newly generated candidate sets.
In a second bottle-relay pair run in reverse order, the workflow took 306.4 → 281.3 s, with 8 → 6 grasp attempts. The two-pair mean was 276.3 → 252.3 s (8.7% less time). Contact outcomes varied between runs at the same seed.
The eight-environment Tianji microbenchmark compared 720 link matrices across three physics states with zero maximum element difference. Looking up three rows and both arm roots took 9.26 → 0.61 ms per control tick (15.2×). This times the read operation alone.
The comparison used the WebRTC-era revision
76ad5aa95and the measured implementation687d0fcae. The 14 changed files are byte-identical in this PR, based on22946b0d3. Six real Chrome replays verified a decoded video, independent ghost selection, Space playback and zero physics steps.All three commands below also ran with the clean package built from
7a7ad9db2and its shipped grasp scoring. Each command collected and read back nine trajectories. Chrome decoded each video and exercised the normal controls.Run the downstream examples
From the DexEChain checkout with the matching sim2real extension installed, activate the Python 3.12 environment and run the following commands. The extension supplies the pinned EmbodiChain revision. The example names select the task deployments. The YAML files select the eight-environment scene-first recipes.
The browser pins the original recording as the solid trajectory. Clicking an augmented candidate toggles only that ghost. Space plays or pauses the recording. For the single-repository UR5 tutorial, follow the tutorial instructions.
Screenshots
The public UR5 tutorial capture from #1244 illustrates the reference and augmented ghost controls.
Type of change
Validation
The focused upstream suite passed all 212 tests, including GPU coverage, and the public API check covered 5,165/5,165 exports. The suite covers branch fallback, geometry matching, per-row feedback, scoped link reads, invalidation and replay. Both focused downstream tests passed, covering the generated CobotMagic deployment and the real Piper five-operator CLI with WebRTC playback and unchanged dataset contents. Existing tests cover the collection and persistence contracts. Focused new cases cover the scheduling and observation behaviors introduced here.
Checklist
black ..