Migrate OVPhysX RigidObject onto OvPhysxView (view series, part 3)#6226
Migrate OVPhysX RigidObject onto OvPhysxView (view series, part 3)#6226AntoineRichard wants to merge 13 commits into
Conversation
bb22e6e to
791b621
Compare
OVPhysX exposes physics attributes as a loose dict of TensorType -> TensorBinding with no view object, unlike Newton's selection.ArticulationView and PhysX's typed tensor views. OvPhysxView wraps the bindings for one prim pattern behind a string-keyed get_attribute / set_attribute surface, addressing attributes by the lowercased TensorType enum name (e.g. "articulation_dof_stiffness"). It needs no Model/State/Control source object because the TensorType already implies where the data lives. Prototype per docs/superpowers/specs/2026-06-17-ovphysx-view-design.md. Adds unit tests covering name<->enum resolution, the read-only guard, discoverability, and get/set dispatch against a fake binding (no native simulation required).
Reworks the view from a convenience wrapper into a layer that can back the OVPhysX asset/data classes, per the PR review of isaac-sim#6224: - read_into(name, dst): zero-copy fill of a caller-owned, possibly structured-dtype buffer (e.g. wp.transformf) via a float32 reinterpret view -- the mechanism the data containers use today. - set_attribute: accepts structured-dtype sources via the same reinterpret; non-float32-width buffers are rejected rather than silently bit-cast. - prim_paths + key_aliases: support the fused multi-prim binding form (create_tensor_binding(prim_paths=[...])) and storing a binding under a different TensorType key, as RigidObjectCollection needs. - binding_for(): raw TensorBinding accessor for adoption. - _CPU_ONLY_NAMES is now derived from tensor_types._CPU_ONLY_TYPES (no drift). - Added joint/tendon/is_fixed_base metadata passthrough; eager construction raises if it creates zero bindings; get_attribute allocates a fresh buffer (no aliasing); nested error hierarchy; PhysX/binding Protocols. Device policy: no implicit CPU<->GPU conversion. CPU-resident property types are read/written on CPU; a buffer on the wrong device raises DeviceMismatch instead of being staged. Device-less host data (numpy/list) is materialized on the binding's native device.
The OvPhysxView addition is a significant new public surface for the OVPhysX backend, so promote the changelog fragment from a minor to a major bump. Reword the entry to describe the binding-manager surface (read_into, the no-device-conversion policy) and drop the internal design-note path from the user-facing changelog.
From the second PR review of isaac-sim#6224: - Critical: _as_binding_view now requires a float32 scalar dtype before the zero-copy reinterpret. A same-byte-width wrong dtype (int32) previously passed the count-only guard and was bit-reinterpreted into garbage on the write path; sub-4-byte dtypes (float16) produced a misleading "0 elements" error. Both are now rejected with a clear message. Regression tests added (verified failing without the guard). - _resolve enforces the str | TensorType union and raises UnknownAttribute on anything else, instead of letting a bogus key reach the wheel. - _binding accesses binding.count directly (a malformed binding surfaces instead of being masked as a phantom no-match) and surfaces the underlying create_tensor_binding exception in the AttributeUnavailable message. - Added docstrings to the six metadata properties; dropped the unused runtime_checkable decorator. - Tests: same-byte/sub-4-byte dtype rejection, get_attribute(out=) wrong device, both indices+mask forwarded, read/write through a prim_paths+key_aliases view, non-str/non-TensorType key, and a read-only-names-are-valid-vocabulary check.
From the API-hardening review of isaac-sim#6224. Validate at the boundary and fail loud instead of silently corrupting, mis-binding, or no-op'ing: - Reject non-contiguous buffers in _as_binding_view (a strided/sliced source would be reinterpreted as contiguous and read/write the wrong memory). - Canonicalize the device (wp.get_device) so a "cuda" view accepts a "cuda:0" buffer instead of raising a spurious, unsatisfiable DeviceMismatch; falls back to the raw string when the device can't be resolved locally. - Reject TensorType.INVALID via the member path too (string path already did). - Normalize key_aliases to TensorType members so string keys are honored rather than silently dropped, and reject aliases that cross the CPU/GPU residency or read-only boundary (the device/read-only guards key on the requested type). - Reject empty pattern/prim_paths and tensor_types-without-eager at construction. - Eager construction with an explicit tensor_types list now surfaces a failing type instead of swallowing it at debug level (default sweep still skips inapplicable types). - Document binding_for as an unguarded escape hatch, get_attribute's native-device return, and the has_attribute name-validity-vs-availability split. Adds regression tests for each (contiguity and INVALID verified failing without the guard).
Surfaced by dogfooding the view in the articulation migration: the assets branch on a "binding or None" pattern for optional/absent bindings (tendon types on a tendon-less articulation, not-yet-created bindings), which the raising binding_for can't express. try_binding_for returns None when the attribute is valid but not available for the view's prims, while still raising UnknownAttribute for an invalid name (a programming error, not an availability question).
…xView read_into now reuses the float32 reinterpret of a destination buffer across calls (keyed by buffer id, with a pointer-staleness guard), so the wheel's object-identity read cache stays warm even when callers hand a structured buffer each step -- they no longer need to maintain their own reinterpret cache. get_attribute returns a typed array for attributes with a known structured layout (transformf for poses, spatial_vectorf for velocities, via a hand-maintained _ATTR_DTYPE map) and flat float32 otherwise. This lets the asset data containers drop their bespoke _get_read_view caching and read structured buffers straight through the view.
791b621 to
4752ad5
Compare
OvPhysxView (and OvPhysxFrameView) live in isaaclab_ovphysx.sim.views, which had no API-docs page. Add the automodule stub and wire it into the isaaclab_ovphysx autosummary so the new binding-manager view shows up in the rendered API reference alongside assets / cloner / physics.
Three fixes from the Part 1 review: - get_attribute (no out): route the freshly allocated buffer through _as_binding_view directly instead of the id()-keyed read cache. A fresh buffer can never hit the cache and would leak one entry (keeping the buffer alive) per call in a step loop; the cache only pays off for a reused out/dst buffer. Add a regression test asserting the no-out path leaves _read_views empty. - Raise a dedicated DtypeMismatch instead of ShapeMismatch when a buffer's scalar element type is not float32, so a dtype error no longer reads as a dimensions error. Update the affected tests and the Raises docstrings. - Make the view Warp-native: drop the fragile __module__ string-match that auto-converted Torch tensors on writes. Callers bridge a Torch tensor with wp.from_torch(t), keeping the device policy explicit and avoiding an optional Torch dependency.
4752ad5 to
398157e
Compare
Documentation/comment clarifications from the isaac-sim#6224 review (no behavior change): - Narrow the documented contract to float32-only: attribute_names/has_attribute and the module docstring now state that a listed name is name-validity, not a dtype-support promise; non-float dtype handling awaits wheel dtype metadata. - Mark _READ_ONLY_NAMES explicitly temporary; name the three access modes (read/write, read-only, write-only) and the wheel access_mode enum that should replace the table. - Document key_aliases as an internal collection adapter, not general public API, pending descriptor metadata. - Make the view test scope explicit: mock API mechanics here; live CPU-only-on-GPU / read-only+write-only / structured read_into coverage lives in the asset-integration tests.
Migrate the OVPhysX RigidObject onto the OvPhysxView binding manager (view migration series, part 3; rebased onto develop). One OvPhysxView owns binding creation (fail-loud eager), the data container is built from it, both _get_binding helpers delegate to it. Reads route through read_into (cached reinterpret; CPU-only mass/COM/inertia keep pinned-host staging) and writes through set_attribute. root_view now returns the OvPhysxView (breaking). Verified on both devices (cpu: 42 passed / 10 xfailed; cuda:0: 41 passed / 2 skipped / 8 xfailed).
398157e to
2ba51af
Compare
…ndings (view series, part 1) (#6224) > **📦 Part 1 of a 5-part series** adding an OvPhysX binding view and migrating the assets onto it: > 1. **This PR** — add `OvPhysxView` (the API) + tests. > 2. Migrate the OvPhysX **Articulation** to `root_view`. > 3. Migrate the **RigidObject**. > 4. Migrate the **RigidObjectCollection** (uses the fused `prim_paths`+`key_aliases` path). > 5. Migrate the **sensors** (pose tracking; contact forces use a separate `ContactBinding` API and may not fully adopt the view). > > Parts 2–5 each depend only on this PR and can be reviewed independently. --- # Description Adds **`OvPhysxView`**, a string-keyed binding-management layer over the OVPhysX tensor bindings, in `isaaclab_ovphysx`. **Why.** OVPhysX is the odd backend out: it exposes physics attributes as a loose `dict[TensorType, TensorBinding]` with **no view object**, whereas Newton has `selection.ArticulationView` and PhysX has typed tensor views. Today callers must hold the right `TensorType` enum member, manage a destination buffer, and create bindings against a USD glob — and the `create_tensor_binding` calls are scattered across the asset classes behind a private `_get_binding`. `OvPhysxView` gives OVPhysX a single, discoverable surface as pleasant as Newton's selection API, and one owner of binding creation/caching that the asset/data classes can delegate to. It is the narrow, independently-useful slice carved out of the (shelved) cross-backend `get_property`/`set_property` effort; the full rationale and decision log live in `docs/superpowers/specs/2026-06-17-ovphysx-view-design.md`. **What it provides** - String-keyed access by lowercased `TensorType` name (auto-derived from the wheel enum — no hand-maintained table); a `TensorType` member is also accepted. - `get_attribute(name, out=)` — fresh-allocates a **typed** array (`wp.transformf` for poses, `wp.spatial_vectorf` for velocities, flat `float32` otherwise; via a maintained `_ATTR_DTYPE` map), no aliasing; **`read_into(name, dst)`** — zero-copy fill of a caller-owned, possibly structured-dtype buffer (`wp.transformf`, …) via a `float32` reinterpret view that is **cached per destination buffer** so the wheel's object-identity read cache stays warm even when callers hand a structured buffer each step; `set_attribute(name, values, indices=/mask=)` (structured sources reinterpreted); and a raw `binding_for(name)` accessor for adoption. - The fused multi-prim form (`prim_paths=[...]` + `key_aliases`) so `RigidObjectCollection` can use it. - Discoverability (`attribute_names`, `available_attributes`, `has_attribute`), metadata passthrough, and a nested error hierarchy (`UnknownAttribute` / `ReadOnlyAttribute` / `AttributeUnavailable` / `ShapeMismatch` / `DeviceMismatch`). **Device policy:** no implicit CPU↔GPU conversion — CPU-resident property types are read/written on CPU, and a buffer on the wrong device raises `DeviceMismatch` rather than being staged. **Scope / status:** this PR adds the view plus 48 mock-based unit tests (no live sim). The asset adoption that replaces `_get_binding`/`_binding_read`/`_binding_write` lives in the follow-up PRs (Articulation #6225, RigidObject #6226), which dogfood the view against a live sim on CPU and GPU and have fed fixes back here (e.g. `try_binding_for`, the read-view cache, typed `get_attribute`). Fixes # — N/A (internal design tracking; surviving slice of the shelved cross-backend view effort). ## Type of change - New feature (non-breaking change which adds functionality) ## Screenshots N/A — no user-facing visual change. ## Checklist - [x] I have read and understood the [contribution guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html) - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [ ] I have made corresponding changes to the documentation <!-- design note updated separately (docs/superpowers/specs/2026-06-17-ovphysx-view-design.md); no rendered API-doc change in this PR yet --> - [x] My changes generate no new warnings - [x] I have added tests that prove my feature works (`test/sim/test_ovphysx_view.py`, 48 tests) - [x] I have added a changelog fragment under `source/<pkg>/changelog.d/` for every touched package (do **not** edit `CHANGELOG.rst` or bump `extension.toml` — CI handles that) - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there
Greptile SummaryThis PR migrates
Confidence Score: 4/5The migration is mechanically correct and the CPU-only staging semantics are preserved; the only annotation gap is a root_view return type that omits None for the pre-init window, which misleads static analysis but does not affect runtime behavior. All six required bindings are eagerly validated at init with descriptive errors; read/write paths are guarded by the view's device, dtype, and shape checks. The annotation mismatch on root_view and the dead _get_binding shim are cosmetic. No data-path logic changes introduce correctness concerns. rigid_object.py for the root_view return type and the now-unreachable _get_binding shim; rigid_object_data.py for the redundant binding_for call in the CPU-only staging branch of _read_binding_into. Important Files Changed
Sequence Diagram%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Caller
participant RigidObject
participant OvPhysxView
participant TensorBinding
participant OvPhysX
Note over RigidObject,OvPhysX: Initialization
RigidObject->>OvPhysxView: OvPhysxView(physx, pattern, device)
loop for each TensorType
RigidObject->>OvPhysxView: binding_for(tt)
OvPhysxView->>OvPhysX: create_tensor_binding(tensor_type, pattern)
OvPhysX-->>OvPhysxView: TensorBinding
OvPhysxView-->>RigidObject: TensorBinding cached
end
Note over Caller,TensorBinding: Write path
Caller->>RigidObject: write_root_link_pose_to_sim_index
RigidObject->>OvPhysxView: set_attribute(RIGID_BODY_POSE, data, indices)
OvPhysxView->>OvPhysxView: device/dtype/shape guards
OvPhysxView->>TensorBinding: write(float32_view, indices)
Note over Caller,TensorBinding: Read path
Caller->>RigidObject: data.root_link_pose_w
RigidObject->>OvPhysxView: read_into(RIGID_BODY_POSE, dst)
OvPhysxView->>OvPhysxView: _read_view reinterpret cache
OvPhysxView->>TensorBinding: read(float32_reinterpret)
Note over Caller,TensorBinding: CPU-only write
Caller->>RigidObject: set_masses_index
RigidObject->>RigidObject: wp.launch + wp.copy to pinned CPU
RigidObject->>OvPhysxView: set_attribute(RIGID_BODY_MASS, cpu_buf, indices)
OvPhysxView->>TensorBinding: write(cpu_float32_view, indices)
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Caller
participant RigidObject
participant OvPhysxView
participant TensorBinding
participant OvPhysX
Note over RigidObject,OvPhysX: Initialization
RigidObject->>OvPhysxView: OvPhysxView(physx, pattern, device)
loop for each TensorType
RigidObject->>OvPhysxView: binding_for(tt)
OvPhysxView->>OvPhysX: create_tensor_binding(tensor_type, pattern)
OvPhysX-->>OvPhysxView: TensorBinding
OvPhysxView-->>RigidObject: TensorBinding cached
end
Note over Caller,TensorBinding: Write path
Caller->>RigidObject: write_root_link_pose_to_sim_index
RigidObject->>OvPhysxView: set_attribute(RIGID_BODY_POSE, data, indices)
OvPhysxView->>OvPhysxView: device/dtype/shape guards
OvPhysxView->>TensorBinding: write(float32_view, indices)
Note over Caller,TensorBinding: Read path
Caller->>RigidObject: data.root_link_pose_w
RigidObject->>OvPhysxView: read_into(RIGID_BODY_POSE, dst)
OvPhysxView->>OvPhysxView: _read_view reinterpret cache
OvPhysxView->>TensorBinding: read(float32_reinterpret)
Note over Caller,TensorBinding: CPU-only write
Caller->>RigidObject: set_masses_index
RigidObject->>RigidObject: wp.launch + wp.copy to pinned CPU
RigidObject->>OvPhysxView: set_attribute(RIGID_BODY_MASS, cpu_buf, indices)
OvPhysxView->>TensorBinding: write(cpu_float32_view, indices)
|
| def _read_binding_into(self, tensor_type: int, dst: wp.array) -> None: | ||
| """Read the OVPhysX TensorBinding for *tensor_type* into *dst*. | ||
|
|
||
| Adapter that replaces PhysX's view-getter pattern: the wheel exposes | ||
| ``binding.read(target)`` rather than a getter returning a wp.array, so | ||
| we read into a flat float32 view of *dst*. CPU-only bindings on a | ||
| non-CPU sim go through a lazily-allocated pinned-host wp.array to | ||
| satisfy the wheel's device match. | ||
| Routes through :meth:`~isaaclab_ovphysx.sim.views.OvPhysxView.read_into`, which | ||
| reinterprets a structured *dst* off the binding shape and reuses that reinterpret | ||
| across calls (keeping the wheel's read cache warm). CPU-only bindings on a non-CPU | ||
| sim are read into a pinned-host staging buffer first (the view does not stage across | ||
| devices), then copied to *dst* on the simulation device. | ||
| """ | ||
| binding = self._bindings[tensor_type] | ||
| if self._check_shapes: | ||
| binding = self._view.binding_for(tensor_type) | ||
| dst_bytes = dst.size * wp.types.type_size_in_bytes(dst.dtype) | ||
| binding_bytes = 4 * math.prod(binding.shape) | ||
| assert dst_bytes >= binding_bytes, ( | ||
| f"_read_binding_into: dst buffer too small for binding {tensor_type!r} " | ||
| f"({dst_bytes} B < {binding_bytes} B). Caller allocated dst with " | ||
| f"shape={tuple(dst.shape)}, dtype={dst.dtype}; binding shape={tuple(binding.shape)}." | ||
| ) | ||
| # Build a flat float32 view of dst matching the binding's shape. | ||
| if dst.dtype == wp.float32: | ||
| view = dst | ||
| else: | ||
| view = wp.array( | ||
| ptr=dst.ptr, | ||
| shape=binding.shape, | ||
| dtype=wp.float32, | ||
| device=str(dst.device), | ||
| copy=False, | ||
| ) | ||
| if tensor_type in TT._CPU_ONLY_TYPES and str(view.device) != "cpu": | ||
| if tensor_type in TT._CPU_ONLY_TYPES and self.device != "cpu": | ||
| staging = self._cpu_staging_buffers.get(tensor_type) | ||
| if staging is None: | ||
| staging = wp.zeros(binding.shape, dtype=wp.float32, device="cpu", pinned=True) | ||
| staging = wp.zeros( | ||
| self._view.binding_for(tensor_type).shape, dtype=wp.float32, device="cpu", pinned=True | ||
| ) | ||
| self._cpu_staging_buffers[tensor_type] = staging | ||
| binding.read(staging) | ||
| self._view.read_into(tensor_type, staging) | ||
| # Copy the flat float32 staging into dst (possibly structured) on the sim device. | ||
| if dst.dtype == wp.float32: | ||
| view = dst | ||
| else: | ||
| view = wp.array(ptr=dst.ptr, shape=staging.shape, dtype=wp.float32, device=str(dst.device), copy=False) | ||
| wp.copy(view, staging) | ||
| else: | ||
| binding.read(view) | ||
| self._view.read_into(tensor_type, dst) |
There was a problem hiding this comment.
Redundant
binding_for call in the staging-allocation branch when check_shapes=False
When check_shapes is False, the shape-check block is skipped and no local binding variable is set. The staging-buffer allocation (line 991) then calls self._view.binding_for(tensor_type) to obtain the shape. The old code fetched binding unconditionally at the top of the function and reused it in both branches. Consider hoisting the binding fetch before the check_shapes guard so both the shape check and the staging allocation share the same cached object.
Clear self._root_view in _invalidate_initialize_callback so a stale/destroyed binding cached by the view is not held across a simulation stop; _initialize_impl rebuilds a fresh OvPhysxView on the next play.
Description
Part 3 of the OVPhysX view-migration series. It migrates the OVPhysX
RigidObjectonto the
OvPhysxViewbinding manager introduced in #6224 (Part 1), mirroring theArticulation migration in #6225 (Part 2).
The series dogfoods
OvPhysxViewon the real asset classes to surface weaknesses and toconsolidate the scattered
create_tensor_binding/binding.read/binding.writecallsbehind one managed surface.
What changed
RigidObject._initialize_implbuilds a singleOvPhysxView(physx, pattern, device)andeagerly creates the rigid-body bindings through it. RigidObject keeps its fail-loud
contract (a missing core type raises a helpful
RuntimeError) by creating each viabinding_forrather than the best-efforttry_binding_forthe articulation uses.RigidObjectDatais constructed from the view; counts come from the view's bindings,and both
_get_bindinghelpers delegate toOvPhysxView.try_binding_for.root_view.set_attribute(...)— pose/velocity/wrench plusthe CPU-only mass/COM/inertia properties (which the asset pre-stages to pinned CPU
buffers, so the source is already on the binding's native device and the view's
no-implicit-staging policy is satisfied). Writes now get the view's read-only/device/
dtype/contiguity guards.
root_view.read_into(...).RigidObjectData._read_binding_intoroutes through the view, which caches the
float32reinterpret per destination bufferand derives the structured layout from the binding shape — this path previously rebuilt
the reinterpret on every read, so it is a strict read-cache improvement. CPU-only reads
keep their pinned-host staging.
RigidObject.root_viewnow returns theOvPhysxViewinstead of a rawdict[TensorType, TensorBinding](breaking — see migration note).Breaking change / migration
RigidObject.root_viewreturns anOvPhysxView, not a dict:root_view[tensor_type]→root_view.try_binding_for(tensor_type)root_view.get_attribute(tensor_type)Type of change
Checklist
pre-commitchecks with./isaaclab.sh --formatsource/isaaclab_ovphysx/changelog.d/)changelog.dfragment(s)Testing
Run kitless; cpu and cuda must be separate invocations (process-global device lock):