Repository navigation
Add pouring tasks, fluid replay and DexUni DFSPH prototype - #790
Conversation
Migrate DexUni example settings to the current namespaced solver interface and preserve the existing cloth contact values. Update the Newton config regression test and document the new mapping contract.
Port the retained MJVBD V2 integration consumers to the current DexUni public interface, while keeping AutoSolver as the Newton default. Preserve namespaced solver options, tutorial calibration, reset/mimic handling, and current task configuration.
Allow geometry-only ContactQuery data for DexUni VBD paths and full impulse/friction data for pure MuJoCo. Add capability tests and GPU end-to-end coverage.
…e-adaptation # Conflicts: # embodichain/lab/sim/cfg/simulation.py # tests/sim/test_cfg.py
Add typed fluid scene ownership, observations and row-local refill; gate batching on native SPH capabilities. Generate and qualify the pouring Task Program in MuJoCo before replaying measured states in an independent SPH Gym. Package physical assets and keep source acceptance separate from fluid metrics. Requires DexSim world-isolated SPH support (d2d36fbcb) for batched scenes.
Use Hybrid by default with FastRT support, and route replay videos through the configured record_camera event. Add separate environment buffers, selected save/discard and configurable cadence to record_camera_data. Explicitly commit the final replay recording. Qualified two complete 744-frame Hybrid videos, both renderer smoke runs, row-local recording reset, 128 focused tests and the live Gym reset regression.
Sample a declared capped revolved profile and fill level on a deterministic lattice. Keep solid clearance separate from the free surface and retain the local template in memory for reset. Remove the default water_fill particle asset, resample the same recipe for spacing overrides, and preserve explicit point imports. Qualify complete two-environment Hybrid pouring and per-environment recordings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[Medium impact] The changes add fluid simulation and recording paths to the shared environment and simulation APIs. Require kinematic robot playback before merging; the bounded-run measurement issue is non-blocking.
|
| if not isinstance(physics, NewtonPhysicsCfg) or solver_type != "dexuni": | ||
| raise ValueError("Fluid replay requires Newton DexUni with DFSPH.") | ||
| if cfg.sim_cfg.physics_cfg.num_substeps != 1: | ||
| raise ValueError( | ||
| "Fluid Gym replay requires num_substeps=1; choose physics_dt for boundary cadence." | ||
| ) |
There was a problem hiding this comment.
FluidReplayEnv accepts joint_mode: dynamic, but playback only writes the robot's current positions and velocities. Reset leaves the drive targets at the first recorded pose. Each physics step can then pull the robot toward that pose instead of following the recording, changing both the robot motion and the boundaries seen by the liquid.
Require a compatible kinematic configuration before creating the scene, or explicitly set up kinematic joint playback.
Prompt To Fix With AI
This is a comment left during a code review.
Path: embodichain/lab/gym/envs/fluid_replay.py
Line: 126-131
Comment:
**Drives fight replayed motion**
`FluidReplayEnv` accepts `joint_mode: dynamic`, but playback only writes the robot's current positions and velocities. Reset leaves the drive targets at the first recorded pose. Each physics step can then pull the robot toward that pose instead of following the recording, changing both the robot motion and the boundaries seen by the liquid.
Require a compatible kinematic configuration before creating the scene, or explicitly set up kinematic joint playback.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| for action in actions: | ||
| yield action | ||
| poses.append(cup.get_local_pose().cpu().numpy().copy()) |
There was a problem hiding this comment.
Diagnostics miss the final pose
During a bounded --seconds run, execute_demo_episode() stops and closes this iterator without resuming past yield action. The last completed step's pose is never captured. Incomplete runs also skip settling, so _cup_qualification() reports the previous step's return error and can miss the largest lift or tilt.
Capture the final measured pose after execution returns, including interrupted runs.
Prompt To Fix With AI
This is a comment left during a code review.
Path: embodichain/lab/scripts/run_fluid_task_program.py
Line: 159-161
Comment:
**Diagnostics miss the final pose**
During a bounded `--seconds` run, `execute_demo_episode()` stops and closes this iterator without resuming past `yield action`. The last completed step's pose is never captured. Incomplete runs also skip settling, so `_cup_qualification()` reports the previous step's return error and can miss the largest lift or tilt.
Capture the final measured pose after execution returns, including interrupted runs.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Move pour acceptance and particle sidecar publication into the ordinary Task Program collection boundary. Keep row-local spooling bounded, remove the standalone fluid runner, and update examples and project context.
Keep the fluid deployment as the single pour-cup entry. Remove the empty-cup environment, program, integration and their exclusive embodiment and execution policy. Update packaging expectations, hand-binding coverage and task documentation. Validation: 57 targeted tests passed; the retained deployment inspector and production task catalog passed. Context: no update needed; existing component ownership and fluid workflow guidance remain accurate.
Integrate main at fcca082, preserving fluid sidecars, generic collection qualification, and main's physical evaluation and managed expert-source support. Resolve six overlapping files without restoring removed helper modules or the empty-cup deployment.
Description
Adapt the pouring task, diagnostic replay and the standalone DexUni robot demonstration to the current fluid foundation in #777. The task now uses DexUni with nested DFSPH configuration, follows the relocated task and robot packages, and records current camera frames at the configured control rate.
Fluid trajectories now use the ordinary Gym replay modes:
kinematicandcontrolrestore recorded robot, rigid-body and full particle positions/velocities without advancing physics;dynamicrestores the initial particle state and replays controller actions through DexUni. The ordinaryrun-taskcollection path now qualifies the pour and publishes bounded particle sidecars with--record-trajectory --record-fluid-state; the separate fluid runner andFluidReplayEnvare removed.The fluid deployment uses an 80-sample minimum, a 25 ms control interval (six 1/240 s physics steps), and a 1.5 rad/s² planning acceleration budget.
PourOptions.rotation_durationnow time-stretches each outward tilt and return leg to at least four seconds when the planned motion is shorter; it no longer fills the requested duration with terminal holds. The authored Franka joint speed caps remain in force. A fluid-only embodiment supplies stronger simulated gripper contact; ordinary Newton deployments keep their calibration. The standalone DexUni demonstration uses the same task policy.A fluid scene without an explicit joint selector assigns the finalized joint topology to MuJoCo before solver construction. This allows the current DexUni/DFSPH prototype to advance the Franka and free cup together while preserving explicit selectors. The task sends both gripper finger targets, avoiding a missed grasp after the newer Gym action-selection changes.
The cup keeps separate rigid-contact hulls and a particle-only SDF boundary. On DexSim builds without
NewtonCollisionDesc.contributes_to_mass, dynamic boundaries require explicit mass and inertia; the task authors the cup COM and inertia from its contact hulls. The FastRT liquid uses a compatible blue material. The Gym runner saves incomplete or rejected videos as diagnostics rather than qualifying them as accepted recordings.Stack and dependencies
mainate06a16477. PR Adapt DexSim 0.6 and consolidate DexUni two-way support #829 and PR Add basic fluid simulation and standalone tutorial #777 are merged.dexsim_engine==0.6.0rc1and runs on Python 3.12 with CUDA on RTX 5090. Its DexUni/DFSPH composition supports isolated worlds and selected-world reset. The earlier 0.5.1 development-build task and performance qualification below remains historical.Validation
2026-11-11
run-taskconsolidation (fab037d34): moved measured cup/received-volume qualification and accepted particle-sidecar publication into the ordinary collection boundary, with row-selected 32 MiB spooling and unique artifact names. Removedrun_fluid_task_program.pyand its standalone tests; the Sim example now imports the shared physical measurement helper. A default 76,607-particle NoRender run passed with 756 control steps, 234.5 mL received, 11.9 cm lift, 79.8° tilt, 1.21 cm return error, and a 1.39 GB sidecar. That source completed full 756-stepkinematic(maximum state error 3.1e-6) anddynamicreplay. A final 3 mm recording with a unique file name and 178 MB sidecar passed qualification and full kinematic replay; FastRT produced exactly one accepted video after removing an initial-reset recording artifact. Focused replay/runner/camera tests passed (101), official task/config/expansion tests passed (253), context tests passed (42), Black 26.10.1, API coverage 4,832/4,832 and context check passed. The read-only Task Program inspector still rejects the existingfluid_objectfield inenv.fluid.yaml; direct config decode and live execution passed. Context review revised the fluid replay overview and MAP ownership.2026-11-11 solver-config follow-up:
e8028a153removes the duplicate standalone-SPH check from native object conversion incfg/physics.py. One normalization guard remains because DexSim registerssphas a named solver; removing it would reopen top-levelsolver_type=sphwhile named providers remain supported. The direct-provider bypass is covered by a new rejection test. The 122 relevant solver/config tests passed (1 skipped), Black 26.10.1 and API coverage 4,833/4,833 passed.2026-11-11 latest-main sync after standalone-SPH cleanup: merged
origin/mainate06a16477incf719624c. Main's new test expected top-levelsolver_type=sphto pass through as a registered SDK solver; this conflicts with the fluid deployment's DexUni-only contract, so the explicit rejection regression was retained. The only content conflict in runtime-control tests was resolved using the unknown-capability fallback fixture. The combined branch passed 106 focused fluid/runtime tests, 42 context tests, Black 26.10.1 and API coverage 4,833/4,833.2026-11-11 standalone-SPH cleanup: commit
7d3d9f572removes the remaining direct production construction ofSPHSolverCfg, obtains fluid defaults throughDexUniSolverCfg(sph_cfg=...), and centralizes access to DexSim's prepared internal fluid component behind the DexUni backend. Positive runtime/config tests now configure fluid only through DexUni. Publicsolver_type=sphand a direct top-levelSPHSolverCfgremain explicitly rejected, with a negative regression test; the native SDK still names DexUni's internal fieldssph_cfgandsph_solver. After rebasing onto the latest pour cleanup, 100 focused CPU tests, 3 native CUDA fluid lifecycle tests, 42 context tests, the fluid deployment inspector, Black 26.10.1, context validation and API coverage 4,833/4,833 passed.2026-11-11 DexUni-only capability cleanup (
07af65fe5): fluid setup no longer loads the standalone SPH runtime to decide whether batched worlds are supported. After scene preparation and before the first physics step, it checks world isolation and selected-world reset on the prepared DexUni fluid component. The native nestedsph_cfgand internalsph_solverremain; directsolver_type: sphstays rejected. On the current PR head, 94 focused CPU fluid/Replay tests, 6 native GPU fluid isolation/runtime/visibility cases, and 42 context tests passed. Black 26.10.1 left 2,463 files unchanged, API coverage is 4,833/4,833, and context validation passed. The existing unified Replay modes and the merge frommainwere retained.2026-11-11 latest-main sync after unified replay: merged
origin/mainat02d0fda91in commit24ddaaf58. The only content conflict was the adjacentSimulationManagerruntime-control methods; both the DexUni kinematic rigid-trajectory registration and main's joint DC motor registration were retained. The merged branch passed 73 focused fluid/replay tests, 42 context tests, 4 joint-motor registration tests, one native GPU fluid-boundary/reset regression, the fluid deployment inspector, Black 26.10.1, context validation and API coverage 4,833/4,833. GitHub reports the PR mergeable; remote CI checks are pending.2026-11-11 unified fluid replay: commit
65e437ddeadds 32 MiB-per-row content-verified particle-state chunks, arena-local coordinates and pre-World qualification checks. A real one-environment FastRT pour passed physical acceptance and recorded 751 control steps with a 1.3 GiB sidecar; fullkinematicanddynamicCLI replays both completed 751 steps. A real frame-200 kinematic particle check had zero position/velocity error, and a 10-step dynamic check wrote fluid state only at reset before DexUni evolved the particles. Focused validation passed 73 replay/fluid/runner tests, 42 context tests, static deployment inspection, Black 26.10.1 and API coverage 4,777/4,777. Representative four-environment I/O (76,607 particles, 747 frames per row) stored 5.49 GB, wrote in 7.65 s and read sequentially in 3.83 s with 220 MiB peak process RSS; this is a synthetic I/O measurement with possible filesystem cache effects, not an isolated solver benchmark. The recorded real run took 117.75 s wall time. An unrecorded baseline was stopped because other GPU jobs contended for the device, so no recording-overhead claim is made. Local evidence is underoutputs/unified-fluid-replay-source-20261111/andoutputs/unified-fluid-replay-io-benchmark-20261111/. Follow-upbc2618e1cscopes the recorder to one published trajectory: it detaches after saving and rejects post-finalization capture so a later reset cannot delete referenced sidecars. The 23 directly affected tests, Black 26.10.1 and API coverage 4,833/4,833 passed after rebasing onto the newer fluid capability fix.2026-11-11 root-cause fix and four-environment qualification: the 1,056-step reference had no visible table splash with the same 1/240 s outer physics step and two DFSPH substeps. The shortened 670-step run completed the nominal three-second tilt segment but its moving trajectory reached the poured pose early and spent the remaining frames holding there. Two four-environment physics-step controls (four substeps, or 1/480 s with 12 physics steps per control) still splashed and reduced mean received volume to 110.0 and 106.4 mL respectively; a smaller solver step was not the remedy. The fix stretches the moving joint trajectory when
rotation_durationexceeds its native duration. With the unchanged six physics steps per 25 ms control update, four FastRT environments at 1.5 mm and 76,607 particles each all passed every segment and physical acceptance: 746 task steps/18.65 simulated seconds, versus 670/16.75 in the splashing variant and 1,056/26.4 task seconds in the reference. Received volume was 170.72–171.23 mL (mean 170.98), maximum tilt 67.59–67.64°, and return error 1.66–1.79 mm. The synchronized four-view recording shows no conspicuous liquid spilled across the table; fluid speed settles after the pour. One-run wall time was 329.2 s. Validation: 10 focused Pour tests (including midpoint motion regression), 42 context tests, static deployment inspection, Black 26.10.1, API coverage 4,778/4,778, and context validation. The report and 18.675 s four-view video are local underoutputs/pr790-true-4s-tilt-four-20261111/. This qualifies DexSim development build0.6.0+88ac9206e0cb9c7894df6c7780f3e157806f0239, not published rc1.Earlier 2026-11-11 slower-pour qualification after main sync (superseded): merged
mainatd95a5f161and retained both the fluid runtime controls and main's joint initialization controls; direct standalonesphremains rejected. On RTX 5090, Python 3.12 and DexSim development build0.6.0+88ac9206e0cb9c7894df6c7780f3e157806f0239, the complete four-environment FastRT Task Program passed all segments and physical acceptance in every row at 1.5 mm with 76,607 particles per environment.PourOptions.rotation_durationrequests at least three seconds for each tilt and return, giving 121 and 120 frames versus 80 and 80 in the fast variant. Task execution was 670 steps/16.75 simulated seconds versus 563 steps/14.075 seconds; the 40 fps recordings have 671 frames. Received volume was 115.89–132.67 mL (mean 120.27 mL versus fast variant mean 111.61 mL); return error was 1.52–1.60 mm. Visible splashing remains. One-run measured wall time was 282.0 s versus 237.6 s. Focused validation passed: 414 simulation/config tests (7 skipped), 27 Pour skill/integration tests, 22 task/package tests, 42 context tests, 11 API-doc tests, static fluid deployment inspection, Black 26.10.1, API coverage (4,778/4,778), context validation and Sphinx dummy build with existing warnings. The report and synchronized four-view video are local underoutputs/pr790-slower-pour-main-sync-four-env-20261111/. This qualifies the cited development build, not published rc1.Earlier 2026-11-11 fast four-environment qualification (superseded): on RTX 5090, Python 3.12 and DexSim development build
0.6.0+88ac9206e0cb9c7894df6c7780f3e157806f0239, the complete four-environment FastRT Task Program passed all segments and physical acceptance in every row at 1.5 mm with 76,607 particles per environment. Control remains six 1/240 s physics steps per 25 ms update. The task used 563 steps and 14.075 simulated seconds versus 1,056 steps and 28.4 seconds before (50.4% less physical time); videos have 564 versus 1,137 frames at 40 fps. Lift was 120.97–121.03 mm, tilt 69.32–69.37°, return error 1.57–1.59 mm, and received volume 106.89–119.66 mL. One-run measured wall time was 237.6 s versus 504.5 s, not an isolated solver benchmark. The fluid-onlyfranka_panda_fast_fluidcomponent uses finger max effort 160 and dynamic friction 3.0; the original Franka joint speed caps and ordinary Newton embodiment remain unchanged. Static inspection for both deployments, 22 focused tests, 42 context tests, Black 26.10.1, API coverage (4,754/4,754), and context validation passed. The report and four recordings are local underoutputs/pr790-four-env-physical-half-20261111/. This qualifies the cited development build, not published rc1.Earlier control-step-only trial (superseded): on the same RTX 5090/Python 3.12/DexSim development build
0.6.0+88ac9206e0cb9c7894df6c7780f3e157806f0239, the full four-environment Task Program again completed all four segments and passed physical acceptance in every row at 1.5 mm and 76,607 particles per environment. Task control steps fell from 1,056 to 533 (49.5% fewer); each FastRT recording fell from 1,137 frames at 40 fps to 574 frames at 20 fps. Simulated duration remained 28.4–28.7 s. Measured received volume was 145.38–147.88 mL, lift 119.35–119.36 mm, maximum tilt 63.45–63.90°, and return error 3.21–3.58 mm. One-run wall time was 497.5 s versus the previous 504.5 s; no wall-time speedup is claimed. The four-view video and report are local underoutputs/pr790-four-env-half-steps-20261111/. Static deployment inspection and 22 related tests passed; Black 26.10.1 left 2,401 files unchanged, API coverage was 4,754/4,754, and context validation passed.2026-11-11 full four-environment pouring qualification: on RTX 5090 with Python 3.12 and DexSim development build
0.6.0+88ac9206e0cb9c7894df6c7780f3e157806f0239, the configured Task Program completed all four segments (pick_cup,aim_cup_above_bowl,pour_water,return_cup) in all four environments and passed measured physical acceptance for every row. At the task default 1.5 mm spacing there were 76,607 particles per environment. Each environment simulated 28.4 s; the full run took 504.5 s wall time and produced four 960x720 FastRT recordings at 40 fps. Measured lift was 120.79–120.80 mm, maximum tilt 63.47–63.49°, return error 2.09–2.11 mm, and received volume 145.16–145.69 mL. The run report and a synchronized four-view video are retained locally underoutputs/pr790-four-env-full-20261111/. This qualification applies to the cited development build; publisheddexsim_engine==0.6.0rc1was not run in this four-environment acceptance pass.2026-11-11 main sync and preview: merged
origin/mainat33a81cd25, resolved four overlaps in API docs and tests, and migrated the two Franka firm-grip embodiments to the current declared gripper binding. Fixed the runner reporttaskpath after recording. Three pour-cup deployments passed static inspection; 280 focused tests passed (4 skipped), 42 context tests and 11 API documentation tests passed. Black 26.10.1 left 2,401 files unchanged; API coverage is 4,754/4,754; context validation and Sphinx dummy build passed. A four-environment FastRT preview ran for 2.05 simulated seconds at 3 mm spacing with 9,778 particles per environment, producing four 960x720, 40 fps diagnostic videos and a local 2x2 preview. The run stopped duringpick_cupby the requested preview duration, so it is not a physical acceptance result.2026-11-10 main sync: merged
origin/mainat981894847into Add pouring tasks, fluid replay and DexUni DFSPH prototype #790 and resolved the overlapping fluid changes and Atomic Skill rename conflicts. 700 focused CPU tests passed (3 skipped), 224 Atomic Skill example tests passed (2 skipped), all three configured deployments passed static inspection, and 5 serial native GPU fluid checks passed. Black left 2,324 files unchanged; API coverage was 4,691/4,691; 42 context tests, context validation and Sphinx dummy build passed. The full pouring Task Program physical acceptance run was not repeated.2026-11-09 target sync: merged
codex/adapt-latest-dexsimatc6e991c2(including merged Add basic fluid simulation and standalone tutorial #777 ate14fc4e78) and resolved seven overlapping fluid conflicts. All three configured deployments passed static inspection; 700 focused CPU tests passed (3 skipped), 42 context tests and 11 API documentation tests passed, and 5 serial native GPU checks passed for online fluid, selected-world reset, diagnostic replay and camera visibility on DexSim 0.6.0rc1. Black left 2,322 files unchanged; API coverage is 4,457/4,457; context validation and the Sphinx dummy build passed with existing documentation warnings. The full Task Program physical acceptance run was not repeated on 0.6.0rc1.Earlier qualification on the DexSim 0.5.1 development build:
black .ran with 26.3.1 and 26.10.1; API coverage is 3300/3300, agent context checks pass, and task discovery listspour_cup.Type of change
Checklist
black ..