Add OvPhysxView: a string-keyed binding manager for OVPhysX tensor bindings (view series, part 1)#6224
Conversation
Reworks the view from a convenience wrapper into a layer that can back the OVPhysX asset/data classes, per the PR review of isaac-sim#6224: - read_into(name, dst): zero-copy fill of a caller-owned, possibly structured-dtype buffer (e.g. wp.transformf) via a float32 reinterpret view -- the mechanism the data containers use today. - set_attribute: accepts structured-dtype sources via the same reinterpret; non-float32-width buffers are rejected rather than silently bit-cast. - prim_paths + key_aliases: support the fused multi-prim binding form (create_tensor_binding(prim_paths=[...])) and storing a binding under a different TensorType key, as RigidObjectCollection needs. - binding_for(): raw TensorBinding accessor for adoption. - _CPU_ONLY_NAMES is now derived from tensor_types._CPU_ONLY_TYPES (no drift). - Added joint/tendon/is_fixed_base metadata passthrough; eager construction raises if it creates zero bindings; get_attribute allocates a fresh buffer (no aliasing); nested error hierarchy; PhysX/binding Protocols. Device policy: no implicit CPU<->GPU conversion. CPU-resident property types are read/written on CPU; a buffer on the wrong device raises DeviceMismatch instead of being staged. Device-less host data (numpy/list) is materialized on the binding's native device.
From the second PR review of isaac-sim#6224: - Critical: _as_binding_view now requires a float32 scalar dtype before the zero-copy reinterpret. A same-byte-width wrong dtype (int32) previously passed the count-only guard and was bit-reinterpreted into garbage on the write path; sub-4-byte dtypes (float16) produced a misleading "0 elements" error. Both are now rejected with a clear message. Regression tests added (verified failing without the guard). - _resolve enforces the str | TensorType union and raises UnknownAttribute on anything else, instead of letting a bogus key reach the wheel. - _binding accesses binding.count directly (a malformed binding surfaces instead of being masked as a phantom no-match) and surfaces the underlying create_tensor_binding exception in the AttributeUnavailable message. - Added docstrings to the six metadata properties; dropped the unused runtime_checkable decorator. - Tests: same-byte/sub-4-byte dtype rejection, get_attribute(out=) wrong device, both indices+mask forwarded, read/write through a prim_paths+key_aliases view, non-str/non-TensorType key, and a read-only-names-are-valid-vocabulary check.
From the API-hardening review of isaac-sim#6224. Validate at the boundary and fail loud instead of silently corrupting, mis-binding, or no-op'ing: - Reject non-contiguous buffers in _as_binding_view (a strided/sliced source would be reinterpreted as contiguous and read/write the wrong memory). - Canonicalize the device (wp.get_device) so a "cuda" view accepts a "cuda:0" buffer instead of raising a spurious, unsatisfiable DeviceMismatch; falls back to the raw string when the device can't be resolved locally. - Reject TensorType.INVALID via the member path too (string path already did). - Normalize key_aliases to TensorType members so string keys are honored rather than silently dropped, and reject aliases that cross the CPU/GPU residency or read-only boundary (the device/read-only guards key on the requested type). - Reject empty pattern/prim_paths and tensor_types-without-eager at construction. - Eager construction with an explicit tensor_types list now surfaces a failing type instead of swallowing it at debug level (default sweep still skips inapplicable types). - Document binding_for as an unguarded escape hatch, get_attribute's native-device return, and the has_attribute name-validity-vs-availability split. Adds regression tests for each (contiguity and INVALID verified failing without the guard).
OVPhysX exposes physics attributes as a loose dict of TensorType -> TensorBinding with no view object, unlike Newton's selection.ArticulationView and PhysX's typed tensor views. OvPhysxView wraps the bindings for one prim pattern behind a string-keyed get_attribute / set_attribute surface, addressing attributes by the lowercased TensorType enum name (e.g. "articulation_dof_stiffness"). It needs no Model/State/Control source object because the TensorType already implies where the data lives. Prototype per docs/superpowers/specs/2026-06-17-ovphysx-view-design.md. Adds unit tests covering name<->enum resolution, the read-only guard, discoverability, and get/set dispatch against a fake binding (no native simulation required).
Reworks the view from a convenience wrapper into a layer that can back the OVPhysX asset/data classes, per the PR review of isaac-sim#6224: - read_into(name, dst): zero-copy fill of a caller-owned, possibly structured-dtype buffer (e.g. wp.transformf) via a float32 reinterpret view -- the mechanism the data containers use today. - set_attribute: accepts structured-dtype sources via the same reinterpret; non-float32-width buffers are rejected rather than silently bit-cast. - prim_paths + key_aliases: support the fused multi-prim binding form (create_tensor_binding(prim_paths=[...])) and storing a binding under a different TensorType key, as RigidObjectCollection needs. - binding_for(): raw TensorBinding accessor for adoption. - _CPU_ONLY_NAMES is now derived from tensor_types._CPU_ONLY_TYPES (no drift). - Added joint/tendon/is_fixed_base metadata passthrough; eager construction raises if it creates zero bindings; get_attribute allocates a fresh buffer (no aliasing); nested error hierarchy; PhysX/binding Protocols. Device policy: no implicit CPU<->GPU conversion. CPU-resident property types are read/written on CPU; a buffer on the wrong device raises DeviceMismatch instead of being staged. Device-less host data (numpy/list) is materialized on the binding's native device.
The OvPhysxView addition is a significant new public surface for the OVPhysX backend, so promote the changelog fragment from a minor to a major bump. Reword the entry to describe the binding-manager surface (read_into, the no-device-conversion policy) and drop the internal design-note path from the user-facing changelog.
From the second PR review of isaac-sim#6224: - Critical: _as_binding_view now requires a float32 scalar dtype before the zero-copy reinterpret. A same-byte-width wrong dtype (int32) previously passed the count-only guard and was bit-reinterpreted into garbage on the write path; sub-4-byte dtypes (float16) produced a misleading "0 elements" error. Both are now rejected with a clear message. Regression tests added (verified failing without the guard). - _resolve enforces the str | TensorType union and raises UnknownAttribute on anything else, instead of letting a bogus key reach the wheel. - _binding accesses binding.count directly (a malformed binding surfaces instead of being masked as a phantom no-match) and surfaces the underlying create_tensor_binding exception in the AttributeUnavailable message. - Added docstrings to the six metadata properties; dropped the unused runtime_checkable decorator. - Tests: same-byte/sub-4-byte dtype rejection, get_attribute(out=) wrong device, both indices+mask forwarded, read/write through a prim_paths+key_aliases view, non-str/non-TensorType key, and a read-only-names-are-valid-vocabulary check.
From the API-hardening review of isaac-sim#6224. Validate at the boundary and fail loud instead of silently corrupting, mis-binding, or no-op'ing: - Reject non-contiguous buffers in _as_binding_view (a strided/sliced source would be reinterpreted as contiguous and read/write the wrong memory). - Canonicalize the device (wp.get_device) so a "cuda" view accepts a "cuda:0" buffer instead of raising a spurious, unsatisfiable DeviceMismatch; falls back to the raw string when the device can't be resolved locally. - Reject TensorType.INVALID via the member path too (string path already did). - Normalize key_aliases to TensorType members so string keys are honored rather than silently dropped, and reject aliases that cross the CPU/GPU residency or read-only boundary (the device/read-only guards key on the requested type). - Reject empty pattern/prim_paths and tensor_types-without-eager at construction. - Eager construction with an explicit tensor_types list now surfaces a failing type instead of swallowing it at debug level (default sweep still skips inapplicable types). - Document binding_for as an unguarded escape hatch, get_attribute's native-device return, and the has_attribute name-validity-vs-availability split. Adds regression tests for each (contiguity and INVALID verified failing without the guard).
Surfaced by dogfooding the view in the articulation migration: the assets branch on a "binding or None" pattern for optional/absent bindings (tendon types on a tendon-less articulation, not-yet-created bindings), which the raising binding_for can't express. try_binding_for returns None when the attribute is valid but not available for the view's prims, while still raising UnknownAttribute for an invalid name (a programming error, not an availability question).
…xView read_into now reuses the float32 reinterpret of a destination buffer across calls (keyed by buffer id, with a pointer-staleness guard), so the wheel's object-identity read cache stays warm even when callers hand a structured buffer each step -- they no longer need to maintain their own reinterpret cache. get_attribute returns a typed array for attributes with a known structured layout (transformf for poses, spatial_vectorf for velocities, via a hand-maintained _ATTR_DTYPE map) and flat float32 otherwise. This lets the asset data containers drop their bespoke _get_read_view caching and read structured buffers straight through the view.
976656f to
5bc49c0
Compare
OvPhysxView (and OvPhysxFrameView) live in isaaclab_ovphysx.sim.views, which had no API-docs page. Add the automodule stub and wire it into the isaaclab_ovphysx autosummary so the new binding-manager view shows up in the rendered API reference alongside assets / cloner / physics.
Greptile SummaryThis PR introduces
Confidence Score: 3/5The new API is well-designed and test coverage is thorough, but the no-out get_attribute branch has a memory-management defect that should be fixed before the follow-up PRs adopt this layer in a live sim. The no-out branch of get_attribute routes a freshly-allocated buf through _read_view on every call — cached entries are never reused and warp copy=False views may keep the base array alive, causing indefinite growth of _read_views in any simulation loop that calls get_attribute for convenience. The fix is a one-line change but should land before the series continues. source/isaaclab_ovphysx/isaaclab_ovphysx/sim/views/ovphysx_view.py — specifically the no-out branch of get_attribute (line 328) and the _as_binding_view dtype-check error type (line 588). Important Files Changed
Reviews (1): Last reviewed commit: "docs(ovphysx): add isaaclab_ovphysx.sim...." | Re-trigger Greptile |
| alloc_shape, dtype = self._attribute_dtype(tt, binding) | ||
| buf = wp.zeros(alloc_shape, dtype=dtype, device=device) | ||
| binding.read(self._read_view(buf, binding)) | ||
| return buf |
There was a problem hiding this comment.
_read_views grows without bound on every no-out structured-dtype get_attribute call
buf is freshly allocated on every call, so id(buf) is unique and the cached float32 reinterpret stored at _read_views[id(buf)] can never be hit on a subsequent call. Because warp's copy=False arrays commonly keep a reference to their base buffer, the float32 view in the dict may also keep buf alive, preventing GC. In a simulation loop that calls view.get_attribute("rigid_body_pose") each step, _read_views accumulates one float32-view (and potentially one structured array) per step, growing indefinitely for the life of the view.
The cache only pays off for a persistent dst that is reused across calls — which is exactly the read_into / get_attribute(out=) path. In the no-out allocation path, _as_binding_view should be called directly to bypass the cache.
| if scalar is not wp.float32: | ||
| raise OvPhysxView.ShapeMismatch( | ||
| f"{role} must have float32 scalar elements (got dtype " | ||
| f"{getattr(arr.dtype, '__name__', arr.dtype)}); the view reinterprets bits, " | ||
| "not values, so a non-float32 dtype would silently corrupt the buffer." | ||
| ) |
There was a problem hiding this comment.
ShapeMismatch is raised here but the actual fault is a dtype mismatch — the array's scalar type is not float32, not that the shape is wrong. A caller who supplies wp.int32 data will see ShapeMismatch in their traceback and assume they got the dimensions wrong when the real issue is the element type. Consider a dedicated error subclass or at minimum reusing OvPhysxViewError with a dtype-focused message.
| if scalar is not wp.float32: | |
| raise OvPhysxView.ShapeMismatch( | |
| f"{role} must have float32 scalar elements (got dtype " | |
| f"{getattr(arr.dtype, '__name__', arr.dtype)}); the view reinterprets bits, " | |
| "not values, so a non-float32 dtype would silently corrupt the buffer." | |
| ) | |
| if scalar is not wp.float32: | |
| raise OvPhysxView.OvPhysxViewError( | |
| f"{role} must have float32 scalar elements (got dtype " | |
| f"{getattr(arr.dtype, '__name__', arr.dtype)}); the view reinterprets bits, " | |
| "not values, so a non-float32 dtype would silently corrupt the buffer." | |
| ) |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| if type(values).__module__.split(".")[0] == "torch": | ||
| return wp.from_torch(values) |
There was a problem hiding this comment.
Detecting PyTorch tensors by string-matching the module name is fragile: it can be fooled by any user class in a package named
torch, and silently skips torch.Tensor subclasses whose __module__ is not torch.*. The standard pattern is an isinstance guard after a lazy import.
| if type(values).__module__.split(".")[0] == "torch": | |
| return wp.from_torch(values) | |
| try: | |
| import torch as _torch # noqa: PLC0415 (lazy import: torch is optional) | |
| if isinstance(values, _torch.Tensor): | |
| return wp.from_torch(values) | |
| except ImportError: | |
| pass |
Three fixes from the Part 1 review: - get_attribute (no out): route the freshly allocated buffer through _as_binding_view directly instead of the id()-keyed read cache. A fresh buffer can never hit the cache and would leak one entry (keeping the buffer alive) per call in a step loop; the cache only pays off for a reused out/dst buffer. Add a regression test asserting the no-out path leaves _read_views empty. - Raise a dedicated DtypeMismatch instead of ShapeMismatch when a buffer's scalar element type is not float32, so a dtype error no longer reads as a dimensions error. Update the affected tests and the Raises docstrings. - Make the view Warp-native: drop the fragile __module__ string-match that auto-converted Torch tensors on writes. Callers bridge a Torch tensor with wp.from_torch(t), keeping the device policy explicit and avoiding an optional Torch dependency.
marcodiiga
left a comment
There was a problem hiding this comment.
A few API-level comments. Overall this looks useful as an IsaacLab-side consolidation layer; my main concern is keeping the public contract narrow until dtype/access metadata comes from the wheel instead of local tables.
|
|
||
| def attribute_vocabulary() -> list[str]: | ||
| """Return every valid attribute name (sorted lowercased ``TensorType`` members).""" | ||
| return sorted(t.name.lower() for t in TensorType if t.name != "INVALID") |
There was a problem hiding this comment.
API concern: this advertises every TensorType as an OvPhysxView attribute, but the implementation is still largely float32-oriented. The tensor surface includes non-float bindings such as rigid_body_disable_simulation (uint8/bool), so attribute_names / has_attribute can imply support for names that get_attribute will allocate/read with the wrong dtype. I think this should either filter to the currently supported subset, or wait on wheel-exposed dtype metadata before exposing the full TensorType vocabulary as supported attributes.
There was a problem hiding this comment.
Good point. Note that in 0.4.13 all 55 TensorType members are float32 — rigid_body_disable_simulation isn't in the wheel yet — so nothing is mis-read today; this is forward-looking. Rather than guess a subset with another local table, I narrowed the documented contract instead: the module docstring + attribute_names/has_attribute now state they report name-validity, not dtype support, and that the view is float32-only pending wheel dtype metadata. Filed TensorBinding.dtype as the upstream ask (design §7 ask 2); once it lands we can filter attribute_names to a dtype-aware subset and drop the guesswork. Confirmed offline you're planning to expose this.
| # in practice. The wheel enum exposes no writability flag today, so this set is | ||
| # hand-maintained. | ||
| # TODO(ovphysx): source this from a wheel writability query (design doc §7 ask 3). | ||
| _READ_ONLY_NAMES: frozenset[str] = frozenset( |
There was a problem hiding this comment.
Can we keep this local read-only list clearly temporary? The wheel has at least three access modes in practice: read/write, read-only, and write-only control tensors. A local _READ_ONLY_NAMES table will drift as TensorType grows, and a future wheel API should probably expose an access_mode enum rather than a boolean is_writable flag.
There was a problem hiding this comment.
Agreed. Marked _READ_ONLY_NAMES explicitly temporary and reframed the upstream ask around an access_mode enum (read/write, read-only, write-only) rather than a boolean is_writable, so write-only control tensors stay distinguishable. It replaces the hand-maintained table (design §7 ask 3).
| prim_paths: An explicit list of fnmatch globs for the fused multi-prim binding | ||
| form (``create_tensor_binding(prim_paths=[...])``). Mutually exclusive with | ||
| ``pattern``. | ||
| key_aliases: Optional mapping ``requested_type -> created_type`` so a binding can |
There was a problem hiding this comment.
key_aliases makes sense for the collection/fused-binding adapter path, but it is a subtle semantic escape hatch: the requested key and the created binding type can differ. That is fine as an internal IsaacLab adapter, but I would avoid presenting it as the long-term public API without descriptor metadata that records the requested key, source tensor type, shape, native device, and access mode. Otherwise callers can reason from the visible key and get different runtime semantics.
There was a problem hiding this comment.
Agreed — documented key_aliases as an internal IsaacLab adapter for the fused-collection binding path, explicitly not general public API (requested key vs created type intentionally differ), and noted that a public form should carry descriptor metadata (requested key, source tensor type, shape, native device, access mode), deferred to wheel metadata.
| These exercise the pure-Python name<->enum logic and the view's get/set/read-into | ||
| dispatch (including the float32 reinterpret of structured buffers and the | ||
| no-implicit-conversion device policy) against a fake ``PhysX`` + fake ``TensorBinding``. | ||
| Full read/write round-trips on a live sim are covered by the asset integration tests. |
There was a problem hiding this comment.
This PR itself appears to add mock/fake-binding coverage only. That is good for the API mechanics, but before this becomes the hot-path root_view implementation I would like the follow-up/live coverage to explicitly exercise CPU-only properties on a GPU sim, write-only/read-only failures, and at least one structured read_into path against a real ovphysx binding. If that is intentionally covered by later asset-integration PRs, maybe make that expectation explicit here so the current test scope is not overstated.
There was a problem hiding this comment.
Made the scope explicit in the test module docstring: these cover the view's API mechanics against mock bindings; live coverage — CPU-only properties on a GPU sim, read-only/write-only failures, and structured read_into against real bindings — is in the asset-integration tests (test_articulation.py / test_rigid_object.py / test_rigid_object_collection.py) that adopt this as root_view.
Documentation/comment clarifications from the isaac-sim#6224 review (no behavior change): - Narrow the documented contract to float32-only: attribute_names/has_attribute and the module docstring now state that a listed name is name-validity, not a dtype-support promise; non-float dtype handling awaits wheel dtype metadata. - Mark _READ_ONLY_NAMES explicitly temporary; name the three access modes (read/write, read-only, write-only) and the wheel access_mode enum that should replace the table. - Document key_aliases as an internal collection adapter, not general public API, pending descriptor metadata. - Make the view test scope explicit: mock API mechanics here; live CPU-only-on-GPU / read-only+write-only / structured read_into coverage lives in the asset-integration tests.
|
For the record (post-merge): the no- |
Description
Adds
OvPhysxView, a string-keyed binding-management layer over the OVPhysX tensor bindings, inisaaclab_ovphysx.Why. OVPhysX is the odd backend out: it exposes physics attributes as a loose
dict[TensorType, TensorBinding]with no view object, whereas Newton hasselection.ArticulationViewand PhysX has typed tensor views. Today callers must hold the rightTensorTypeenum member, manage a destination buffer, and create bindings against a USD glob — and thecreate_tensor_bindingcalls are scattered across the asset classes behind a private_get_binding.OvPhysxViewgives OVPhysX a single, discoverable surface as pleasant as Newton's selection API, and one owner of binding creation/caching that the asset/data classes can delegate to. It is the narrow, independently-useful slice carved out of the (shelved) cross-backendget_property/set_propertyeffort; the full rationale and decision log live indocs/superpowers/specs/2026-06-17-ovphysx-view-design.md.What it provides
TensorTypename (auto-derived from the wheel enum — no hand-maintained table); aTensorTypemember is also accepted.get_attribute(name, out=)— fresh-allocates a typed array (wp.transformffor poses,wp.spatial_vectorffor velocities, flatfloat32otherwise; via a maintained_ATTR_DTYPEmap), no aliasing;read_into(name, dst)— zero-copy fill of a caller-owned, possibly structured-dtype buffer (wp.transformf, …) via afloat32reinterpret view that is cached per destination buffer so the wheel's object-identity read cache stays warm even when callers hand a structured buffer each step;set_attribute(name, values, indices=/mask=)(structured sources reinterpreted); and a rawbinding_for(name)accessor for adoption.prim_paths=[...]+key_aliases) soRigidObjectCollectioncan use it.attribute_names,available_attributes,has_attribute), metadata passthrough, and a nested error hierarchy (UnknownAttribute/ReadOnlyAttribute/AttributeUnavailable/ShapeMismatch/DeviceMismatch).Device policy: no implicit CPU↔GPU conversion — CPU-resident property types are read/written on CPU, and a buffer on the wrong device raises
DeviceMismatchrather than being staged.Scope / status: this PR adds the view plus 48 mock-based unit tests (no live sim). The asset adoption that replaces
_get_binding/_binding_read/_binding_writelives in the follow-up PRs (Articulation #6225, RigidObject #6226), which dogfood the view against a live sim on CPU and GPU and have fed fixes back here (e.g.try_binding_for, the read-view cache, typedget_attribute).Fixes # — N/A (internal design tracking; surviving slice of the shelved cross-backend view effort).
Type of change
Screenshots
N/A — no user-facing visual change.
Checklist
pre-commitchecks with./isaaclab.sh --formattest/sim/test_ovphysx_view.py, 48 tests)source/<pkg>/changelog.d/for every touched package (do not editCHANGELOG.rstor bumpextension.toml— CI handles that)CONTRIBUTORS.mdor my name already exists there