Skip to content

Support custom Newton providers with a standalone tutorial - #844

Merged
yuecideng merged 51 commits into
mainfrom
codex/simple-newton-solver
Nov 11, 2026
Merged

yuecideng merged 51 commits into
mainfrom
codex/simple-newton-solver

Conversation

@yuecideng

@yuecideng yuecideng commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Description

Support DexSim custom Newton providers through NewtonPhysicsCfg while keeping the spring-slider implementation entirely in scripts/tutorials/sim/simple_newton_solver/. The tutorial contains the provider, two-environment demo, URDF, README, and a Sphinx walkthrough. It is neither a packaged EmbodiChain solver nor a dexsim.newton_solvers entry point.

Direct ProviderNewtonSolverCfg(provider_factory=...) configuration passes through without entry-point discovery. Installed named extensions remain supported by the generic config boundary. DexUni stays the default, and gradient mode still rejects a built-in non-differentiable solver. Articulation initialization queries Scene.dynamics_capabilities() so effort-only providers do not receive unsupported position targets; position-capable solvers retain their hold and mimic targets.

The tutorial's native and functional paths share one semi-implicit spring-slider transition. Its headless demo checks parity, isolated differentiable rollout, effort and parameter gradients, publication, and selective reset. --preview-path writes a trajectory PNG. --render-path captures initial and post-replay RGB images from two EmbodiChain offscreen cameras; --video-path records their 30-step motion as a side-by-side H.264 MP4. --viser also exports both slider meshes to the browser for interactive inspection.

PR #829 has merged. This branch includes main at 33a81cd25f367b4968b71423252af20873c8e8c8.

Scope and dependencies

The tutorial solver accepts independent dynamic prismatic leaves with fixed ancestors and zero gravity. It rejects moving-body collisions, particles, mimic constraints, friction, and target drives, and runs eagerly without CUDA Graph capture. The optional tutorial needs a DexSim build providing ProviderNewtonSolverCfg, FunctionalDynamics, Scene.dynamics_capabilities(), and explicit solver capabilities. The released dependency pin is unchanged; the README explains use with a compatible development SDK.

Type of change

  • Enhancement: direct and named custom Newton provider configuration.
  • Bug fix: capability-aware effort-only initialization and selective reset.
  • Documentation: tutorial-only solver, runnable walkthrough, and PNG preview.

Validation

Candidate: eb954633c.

  • CPU and CUDA tutorial runs passed parity, gradients, publication, and partial-reset checks; both produced published positions [0.0208730716, 0.0313096046], with only the first slider reset to zero. CPU also generated the PNG preview. The Viser run exported two articulation meshes and completed a 30-step replay at positions [0.3481353, 0.5222028]. The new FastRT offscreen path captured both environments before and after the replay as native 640×480 RGB images; the resulting four-panel image was visually inspected. A 40-frame, 4-second, 1280×480 H.264 offscreen video was generated and checked with ffprobe; combined PNG+MP4 output passed in one run.
  • Focused core selection: 144 passed, 35 skipped. Its one failing new test asserted object identity that DexSim does not promise; after correcting the assertion, both affected configuration cases passed. The full selection was not rerun.
  • Agent context tests: 42 passed; API documentation checker tests: 11 passed; context map check passed.
  • black==26.10.1 black .: 2393 files unchanged. API documentation coverage: 4746/4746 exports. Whitespace check passed.
  • Sphinx dummy build completed. The new tutorial page had zero docutils system messages; the repository build retains unrelated warnings.

No tutorial implementation tests are included in the production suite. The core regression test covers direct-provider configuration without package discovery; the runnable tutorial verifies its own numerical and gradient outputs.

Checklist

  • I ran black ..
  • I reviewed affected agent context and updated the physics ownership guidance; robot preset guidance remains accurate.
  • Public API documentation is aligned after removing the packaged example export.
  • I reviewed coverage and the necessity of the remaining test.
  • Dependency behavior is documented; the released pin is unchanged.

xiemenghong and others added 22 commits September 17, 2026 09:48
Migrate DexUni example settings to the current namespaced solver interface and preserve the existing cloth contact values. Update the Newton config regression test and document the new mapping contract.
Port the retained MJVBD V2 integration consumers to the current DexUni public interface, while keeping AutoSolver as the Newton default. Preserve namespaced solver options, tutorial calibration, reset/mimic handling, and current task configuration.
Allow geometry-only ContactQuery data for DexUni VBD paths and full impulse/friction data for pure MuJoCo. Add capability tests and GPU end-to-end coverage.
…e-adaptation

# Conflicts:
#	embodichain/lab/sim/cfg/simulation.py
#	tests/sim/test_cfg.py
@yuecideng yuecideng added docs Documentation, API references and example explanations. enhancement Improvements to existing capabilities; use feature for new capabilities. physics Physics backends, dynamics, contacts, fluids, deformables and physics solvers. kinematics FK, IK, Jacobians, reachability and kinematic solvers; not physics solvers. dexsim DexSim SDK interfaces, version adaptation and integration. labels Oct 22, 2026
@yuecideng
yuecideng marked this pull request as ready for review October 22, 2026 17:37
@greptile-apps

greptile-apps Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge, with a non-blocking fix recommended for large mesh indices.

Fix All in CodexFindings

  1. P2 Large indices select wrong vertices ▶
Fix with agent prompt
### Issue 1
embodichain/lab/sim/cfg/deformable.py:304
`VolumeDeformableObjectCfg` converts tetrahedron indices to `int32` before checking their range. An `int64` index such as `4294967296` becomes `0`, so the later bounds check accepts it and builds a different mesh instead of rejecting the input. Authored surface triangles in `spawn/descriptors.py` have the same problem.

Check the original indices before narrowing either array.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds a spring-slider Newton provider with direct-factory and optional registered-name configuration.

  • Newton callers can run a spring-slider solver natively or as a differentiable rollout.
  • Newton configurations and examples use DexUni’s solver options and scene-based routing.
  • Articulations follow solver capabilities when setting targets and resetting state.
  • Deformable objects accept authored tetrahedra and expanded cloth and particle settings.
  • Contact sensors distinguish measured forces from geometry-only contact rows.
  • Simulation managers can register per-environment runtime controls and particle schedules.
  • Planners can still read collision geometry when contact response is disabled.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Config[NewtonPhysicsCfg] --> Factory[Direct provider factory]
  Config --> Named[Registered provider name]
  Factory --> Provider[Fresh provider per model]
  Named --> Provider
  Provider --> Native[Native stepping]
  Provider --> Functional[Functional rollout]
  Native --> Transition[Shared slider transition]
  Functional --> Transition
  Transition --> Publish[Optional detached Scene publication]
Loading

Reviews (5) · Last reviewed commit: "Merge latest DexSim adaptation and main ..." · Reviewed by Greptile

Synchronize with main at 0845cdd. Resolve placement geometry and collision-test conflicts while preserving authored local-Y pan geometry, main's orthonormal grasp frame, mesh-centering and support-surface checks. Align the new placement audit with the local-Y basin mask and retain regression coverage for a low handle.

Validation: black 26.10.1 (1908 files unchanged); 2086 focused CPU regressions passed, 65 deselected; placement/collision conflict checks 23 passed; API docs 3514/3514; context map validation and context tests passed. Existing simulation, motion and Atomic context remains accurate and sufficient, so no additional context edits are needed. Native GPU/renderer runs were outside this merge validation scope.
Synchronize main at 6881882 with the open environment's DexSim 0.6.0+c3e92584 build. Read solver contact reports through the new API, retaining per-row impulse validity alongside main's contact cache, observer lifecycle and fused loss counters. Forward raw MuJoCo contact parameters through sparse configuration and document the geometry-detection/report distinction. Require dexsim_engine>=0.6.0,<0.7.0.

Validation: 3474 focused CPU regressions passed; 17 CUDA sampling tests passed; 2 native DexUni coupling tests passed; 10 native contact/support/camera checks passed (the support fetch hook was updated and rerun separately). Black 26.10.1: 2215 files unchanged. API docs: 4286/4286. Context checks and tests passed. All 146 referenced DexSim import symbols resolved. Wheel built and verified against worktree bytes and installed DexSim version. Hybrid headless tests used SDL_VIDEODRIVER=dummy; shared conda packages were not reinstalled. The complete example scene matrix and interactive windows were not rerun.
Keep gravity compensation and opt-in mimic stabilization as the supported Newton extensions. Remove seven unused root fields, generic SDK option forwarding and articulation-wide SDF rewriting. Leave native creation/geometry choices at source or SDK defaults and update the owning context.

Reuse existing configuration, source overlay and mimic coverage; reject withdrawn fields at both public decode boundaries and roundtrip non-default values for retained fields. Remove obsolete capability-forwarding cases. Validation on conda open / DexSim 0.6: 724 CPU tests passed, 7 deselected; 2 native two-way coupling tests passed with and without CUDA Graphs. Black 26.10.1: 2215 files unchanged; API docs 4286/4286; context checks/tests and final wheel/source metadata validation passed.
Use the current DexUni configuration contract directly. Remove AutoSolver aliases, legacy flat-option translation and attribute injection, SDK-version probing, and stale solver guards. Preserve native default routing, grouped expert options and contact_mode=auto. Update configuration regression coverage and project context.

Validation: 728 focused CPU tests passed, 20 deselected; black 26.10.1 left 2215 files unchanged; API docs cover 4286/4286 exports; context validation and routing checks pass. The rebuilt wheel matches the changed source and accepts the installed DexSim 0.6.0+c3e92584 build. Native GPU selections were not rerun for this configuration cleanup.
Replace retired solver terminology throughout the tracked documentation and tests. Document the default DexUni family separately from scene ownership and contact_mode=auto, migrate the example's iteration options into vbd_options, and correct path-specific contact-report guidance. Cover the explicit automatic contact policy and native ownership defaults using existing configuration cases.

Validation: 738 focused CPU/docs regressions passed, 20 deselected; nine Python snippets parse, and updated Python/YAML configurations convert to native DexUni automatic defaults. Black 26.10.1, API documentation coverage (4286/4286), context validation and whitespace checks pass. A tracked-file scan finds no retired solver names. The Sphinx build is retried after this commit because the architecture generator requires committed source.
Merge the direct/named SimpleSolverProvider integration onto PR #829 while
preserving DexUni defaults, grouped solver options and strict name validation.
Use the current extension capability contract, including explicit particle,
control and native acceleration support. Retain position holds for capable
solvers and support effort-only initialization and selective reset.

Validate named discovery with isolated distribution metadata so the conda open
installation remains unchanged. Document running this worktree via PYTHONPATH.

Validation (conda open, DexSim 0.6.0+c3e92584, Newton 1.7.0, Warp 1.17.0):
- Provider/config/backend helper tests: 84 passed, including CPU/CUDA parity,
  gradients, named/direct factories and native selective resets.
- Articulation/config/backend regression: 142 passed, 35 GPU cases skipped.
- CPU and CUDA two-environment demos: native/functional parity and gradients.
- Context tests: 42 passed; context validation and API docs coverage passed.
- Black 26.10.1 and diff whitespace checks passed.

Context: revise physics configuration ownership for extension selection and
functional dynamics; robot preset selection remains unchanged.
@yuecideng yuecideng changed the title Add a simple Newton solver with direct provider factories and functional rollout Add a simple Newton provider with latest DexSim extension APIs Nov 7, 2026
@yuecideng
yuecideng changed the base branch from main to codex/adapt-latest-dexsim November 7, 2026 08:49
Keep reset-time contact observations readable until the next solved report, require eager native acquisition outside caller-owned CUDA capture, and normalize Newton impulses with the internal solver duration while retaining World-step history clocks.

Cover real articulation resets, native graph consumers, multi-substep support forces, variable World intervals and history clocks; update sensor guidance and align timeline reset test buffers with quantity validity.
Synchronize origin/main at 460939c. Resolve deformable configuration, Spawn descriptor, example, metadata and context conflicts while retaining explicit tetrahedral meshes, cloth extensions and the native contact sampling fixes.

Validation in conda open: 737 focused regressions, 147 integration regressions and 15 native physics regressions passed. Black 26.10.1 leaves all 2301 files unchanged; API coverage is 4440/4440 and context validation passes.
@yuecideng
yuecideng changed the base branch from codex/adapt-latest-dexsim to main November 9, 2026 04:56
@yuecideng
yuecideng marked this pull request as ready for review November 9, 2026 04:57
raise TypeError("tetrahedra must contain integer indices.")
if raw.ndim != 2 or raw.shape[1:] != (4,) or len(raw) == 0:
raise ValueError("tetrahedra must have non-empty shape (M, 4).")
self.tetrahedra = raw.astype(np.int32)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Large indices select wrong vertices

VolumeDeformableObjectCfg converts tetrahedron indices to int32 before checking their range. An int64 index such as 4294967296 becomes 0, so the later bounds check accepts it and builds a different mesh instead of rejecting the input. Authored surface triangles in spawn/descriptors.py have the same problem.

Check the original indices before narrowing either array.

Prompt To Fix With AI
This is a comment left during a code review.
Path: embodichain/lab/sim/cfg/deformable.py
Line: 304

Comment:
**Large indices select wrong vertices**

`VolumeDeformableObjectCfg` converts tetrahedron indices to `int32` before checking their range. An `int64` index such as `4294967296` becomes `0`, so the later bounds check accepts it and builds a different mesh instead of rejecting the input. Authored surface triangles in `spawn/descriptors.py` have the same problem.

Check the original indices before narrowing either array.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code

# Conflicts:
#	agent_context/topics/simulation-system/physics-configuration.md
#	docs/source/api_reference/embodichain/embodichain.lab.sim.cfg.rst
#	embodichain/lab/sim/cfg/deformable.py
#	embodichain/lab/sim/spawn/descriptors.py
#	examples/sim/atomic_skill/axis_align.py
#	examples/sim/atomic_skill/coordinated_placement.py
#	examples/sim/atomic_skill/move_held_object_cup.py
#	examples/sim/atomic_skill/pickup.py
#	examples/sim/atomic_skill/place.py
#	examples/sim/atomic_skill/pour.py
#	extensions/embodichain_benchmarks/motion_generation/scenarios/_placement_geometry.py
#	extensions/embodichain_benchmarks/motion_generation/scenarios/_placement_qualification.py
#	scripts/tutorials/sim/open_drawer.py
#	tests/examples/atomic_skill/test_run.py
#	tests/examples/atomic_skill/test_support.py
#	tests/sim/test_deformable_cfg.py
#	tests/sim/test_sim_manager.py
@yuecideng yuecideng changed the title Add a simple Newton provider with latest DexSim extension APIs Support custom Newton providers with a standalone tutorial Nov 11, 2026
@yuecideng
yuecideng merged commit f74a1e2 into main Nov 11, 2026
5 of 7 checks passed
@yuecideng
yuecideng deleted the codex/simple-newton-solver branch November 11, 2026 06:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Incorrect existing behavior or a fix restoring the intended behavior. dexsim DexSim SDK interfaces, version adaptation and integration. docs Documentation, API references and example explanations. feature New user-facing capabilities. physics Physics backends, dynamics, contacts, fluids, deformables and physics solvers.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant