Skip to content

RealHand: L6, O6, and L20 dexterous hands - #1556

Open
huadongzhang97 wants to merge 1 commit into
NVIDIA:mainfrom
RealHand-Robotics:realhand-dexterous-hands
Open

huadongzhang97 wants to merge 1 commit into
NVIDIA:mainfrom
RealHand-Robotics:realhand-dexterous-hands

Conversation

@huadongzhang97

@huadongzhang97 huadongzhang97 commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds L6, O6, and L20 dexterous-hand configurations for the official DexHandRetargeter, independent of the arm or glove provider.

  • Packages mesh-free left/right URDFs and vector-retargeting YAMLs.
  • Includes an optional standard OpenXR hand-tracking and controller-trigger example using the official JointRateLimiter.
  • Adds documentation and tests for joint ordering, limits, coupling, solver behavior, and pipeline handling.

Type of change

  • New feature (non-breaking change which adds functionality)

Testing

  • 71 example tests passed on Linux with Python 3.11, including the official dex-retargeting solver.
  • Wheel and CMake install contents verified, including all six URDFs and six YAMLs.
  • Installed-wheel dry runs passed for all three hands in both input modes.
  • Clean documentation build passed with warnings treated as errors.
  • SKIP=check-copyright-year pre-commit run --all-files passed.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the linter and formatter with SKIP=check-copyright-year pre-commit run --all-files
  • I have made corresponding changes to the documentation
  • I have added tests that prove my feature works
  • I have signed off all my commits (git commit -s) per the DCO

Summary by CodeRabbit

  • New Features
    • Added an optional RealHand hand-retargeting example for L6, O6, and L20 models, supporting hand tracking and controller input.
    • Added setup guidance, packaged model configurations, and a dry-run mode to preview joint outputs without starting a session.
  • Tests
    • Added coverage for model configurations, retargeting behavior, pipeline outputs, and dry-run execution.

Add packaged hand kinematics, official DexHandRetargeter configurations,
an optional example, documentation, and focused tests.

Signed-off-by: Huadong Zhang <jpoezhang@gmail.com>
@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

This change adds an opt-in RealHand example for L6, O6, and L20 hands. It packages side-specific URDF models and vector-retargeting configurations, and adds model profile and hand configuration APIs. The example builds controller or hand-tracking pipelines with joint rate limiting, and can run a CloudXR session or print pipeline joint names in dry-run mode. CMake, documentation, and Python tests are added for the example.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant build_pipeline
  participant DexHandRetargeter
  participant JointRateLimiter
  participant CloudXR
  CLI->>build_pipeline: Select model and input mode
  build_pipeline->>DexHandRetargeter: Configure left and right hand retargeters
  DexHandRetargeter->>JointRateLimiter: Pass hand joint targets
  JointRateLimiter->>build_pipeline: Return rate-limited hand outputs
  CLI->>CloudXR: Launch session unless dry-run is selected
Loading
Merge Risk: Minimal · up to d810a

This adds an opt-in RealHand L6, O6 and L20 hand retargeting example, disabled by default, so existing builds and users are unaffected. No concrete defect was found. The one open question is whether the packaged L6 model should enforce the physical hand's finger coupling, which the owners can confirm separately.

Security architecture risk: Minimal · up to d810a

The inspected flow adds optional hand-target computation without changing shared runtime implementations or directly commanding hardware. Input validation, separate left/right state, and the existing rate limiter constrain the new controller path. No introduced or worsened security or failure-containment concern was established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — For the supplied standalone CLI, control over incoming hand/controller samples affects the selected session's computed left/right finger targets and printed output. Direct hardware actuation would require an additional consumer not configured by this example.

Trust Boundaries and Controls

  • observed — The controller converter validates posture shape and finiteness, reads only the trigger value, rejects non-finite trigger updates, and clips finite triggers to the interval from zero to one. Each side owns separate state, and both output branches retain the shared downstream rate limiter.

Resilience and Maintainability Implications

  • observed — Missing or non-finite controller input preserves the previous posture. Reset clears that state before processing the current input, so valid input on the reset frame can immediately establish a new posture. The existing limiter also resets its history and treats its first valid frame as a baseline; this is not an emergency-stop or hardware interlock contract.
Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: adding RealHand L6, O6, and L20 dexterous hand configurations and retargeting support. The raw summary covers packaged URDFs and YAML configs, an optional example, doc…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: adding RealHand L6, O6, and L20 dexterous hand configurations and retargeting support. The raw summary covers packaged URDFs and YAML configs, an optional example, docs, and tests for these three hands, which matches the title's scope.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 9fb23f6 and d810ab2.

Files selected for processing: 33

  • CMakeLists.txt
  • docs/source/references/retargeting/index.rst
  • docs/source/references/retargeting/realhand.rst
  • examples/CMakeLists.txt
  • examples/realhand_hands/CMakeLists.txt
  • examples/realhand_hands/README.md
  • examples/realhand_hands/pyproject.toml
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/__init__.py
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/_controller_posture.py
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/_profiles.py
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/README.md
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/l20/left.urdf
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/l20/left.yml
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/l20/right.urdf
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/l20/right.yml
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/l6/left.urdf
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/l6/left.yml
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/l6/right.urdf
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/l6/right.yml
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/o6/left.urdf
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/o6/left.yml
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/o6/right.urdf
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/config/realhand/o6/right.yml
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/hand_config.py
  • examples/realhand_hands/python/isaaccapture_examples/realhand_hands/hand_example.py
  • tests/python/CMakeLists.txt
  • tests/python/examples/realhand_hands/CMakeLists.txt
  • tests/python/examples/realhand_hands/_kinematics.py
  • tests/python/examples/realhand_hands/_samples.py
  • tests/python/examples/realhand_hands/conftest.py
  • tests/python/examples/realhand_hands/pyproject.toml
  • tests/python/examples/realhand_hands/test_dex_config.py
  • tests/python/examples/realhand_hands/test_pipeline.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants