Skip to content

Migrate OVPhysX PVA sensor onto OvPhysxView (view series, part 5)#6229

Open
AntoineRichard wants to merge 13 commits into
isaac-sim:developfrom
AntoineRichard:antoiner/feat/ovphysx_pva_view
Open

Migrate OVPhysX PVA sensor onto OvPhysxView (view series, part 5)#6229
AntoineRichard wants to merge 13 commits into
isaac-sim:developfrom
AntoineRichard:antoiner/feat/ovphysx_pva_view

Conversation

@AntoineRichard

Copy link
Copy Markdown
Collaborator

🔗 Stacked on top of #6224 (Part 1 — OvPhysxView)

Branched off #6224 as a sibling of the asset migrations (#6225/#6226/#6227). Targets develop; until #6224 merges, the diff below includes the Part 1 OvPhysxView commits. Please review #6224 first.

Description

Part 5 of the OVPhysX view-migration series — route the PVA sensor's tensor-binding management through OvPhysxView. Internal refactor; no public API change (sensors do not expose a root_view).

  • _initialize_impl builds one OvPhysxView instead of separate create_tensor_binding calls; binding handles come from binding_for.
  • Pose/state reads route through read_into (cached float32 reinterpret, typed off the binding shape), replacing the manual wp.array(ptr=...) reinterpret views.
  • The CPU-only RIGID_BODY_COM_POSE keeps its pinned-host staging + wp.copy to the sim device.
  • Numeric behavior unchanged.

Type of change

  • This change requires no public API change (internal refactor)

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • My changes generate no new warnings
  • I have added a changelog fragment (.skip — internal refactor, no user-facing change / no bump)

Testing

test_pva.py — cpu: 13 passed, cuda: 13 passed.

OVPhysX exposes physics attributes as a loose dict of TensorType ->
TensorBinding with no view object, unlike Newton's
selection.ArticulationView and PhysX's typed tensor views. OvPhysxView
wraps the bindings for one prim pattern behind a string-keyed
get_attribute / set_attribute surface, addressing attributes by the
lowercased TensorType enum name (e.g. "articulation_dof_stiffness").
It needs no Model/State/Control source object because the TensorType
already implies where the data lives.

Prototype per docs/superpowers/specs/2026-06-17-ovphysx-view-design.md.
Adds unit tests covering name<->enum resolution, the read-only guard,
discoverability, and get/set dispatch against a fake binding (no native
simulation required).
Reworks the view from a convenience wrapper into a layer that can back the
OVPhysX asset/data classes, per the PR review of isaac-sim#6224:

- read_into(name, dst): zero-copy fill of a caller-owned, possibly
  structured-dtype buffer (e.g. wp.transformf) via a float32 reinterpret
  view -- the mechanism the data containers use today.
- set_attribute: accepts structured-dtype sources via the same reinterpret;
  non-float32-width buffers are rejected rather than silently bit-cast.
- prim_paths + key_aliases: support the fused multi-prim binding form
  (create_tensor_binding(prim_paths=[...])) and storing a binding under a
  different TensorType key, as RigidObjectCollection needs.
- binding_for(): raw TensorBinding accessor for adoption.
- _CPU_ONLY_NAMES is now derived from tensor_types._CPU_ONLY_TYPES (no drift).
- Added joint/tendon/is_fixed_base metadata passthrough; eager construction
  raises if it creates zero bindings; get_attribute allocates a fresh buffer
  (no aliasing); nested error hierarchy; PhysX/binding Protocols.

Device policy: no implicit CPU<->GPU conversion. CPU-resident property types
are read/written on CPU; a buffer on the wrong device raises DeviceMismatch
instead of being staged. Device-less host data (numpy/list) is materialized
on the binding's native device.
The OvPhysxView addition is a significant new public surface for the OVPhysX
backend, so promote the changelog fragment from a minor to a major bump.
Reword the entry to describe the binding-manager surface (read_into, the
no-device-conversion policy) and drop the internal design-note path from the
user-facing changelog.
From the second PR review of isaac-sim#6224:

- Critical: _as_binding_view now requires a float32 scalar dtype before the
  zero-copy reinterpret. A same-byte-width wrong dtype (int32) previously passed
  the count-only guard and was bit-reinterpreted into garbage on the write path;
  sub-4-byte dtypes (float16) produced a misleading "0 elements" error. Both are
  now rejected with a clear message. Regression tests added (verified failing
  without the guard).
- _resolve enforces the str | TensorType union and raises UnknownAttribute on
  anything else, instead of letting a bogus key reach the wheel.
- _binding accesses binding.count directly (a malformed binding surfaces instead
  of being masked as a phantom no-match) and surfaces the underlying
  create_tensor_binding exception in the AttributeUnavailable message.
- Added docstrings to the six metadata properties; dropped the unused
  runtime_checkable decorator.
- Tests: same-byte/sub-4-byte dtype rejection, get_attribute(out=) wrong device,
  both indices+mask forwarded, read/write through a prim_paths+key_aliases view,
  non-str/non-TensorType key, and a read-only-names-are-valid-vocabulary check.
From the API-hardening review of isaac-sim#6224. Validate at the boundary and fail loud
instead of silently corrupting, mis-binding, or no-op'ing:

- Reject non-contiguous buffers in _as_binding_view (a strided/sliced source would
  be reinterpreted as contiguous and read/write the wrong memory).
- Canonicalize the device (wp.get_device) so a "cuda" view accepts a "cuda:0"
  buffer instead of raising a spurious, unsatisfiable DeviceMismatch; falls back
  to the raw string when the device can't be resolved locally.
- Reject TensorType.INVALID via the member path too (string path already did).
- Normalize key_aliases to TensorType members so string keys are honored rather
  than silently dropped, and reject aliases that cross the CPU/GPU residency or
  read-only boundary (the device/read-only guards key on the requested type).
- Reject empty pattern/prim_paths and tensor_types-without-eager at construction.
- Eager construction with an explicit tensor_types list now surfaces a failing
  type instead of swallowing it at debug level (default sweep still skips
  inapplicable types).
- Document binding_for as an unguarded escape hatch, get_attribute's native-device
  return, and the has_attribute name-validity-vs-availability split.

Adds regression tests for each (contiguity and INVALID verified failing without
the guard).
Surfaced by dogfooding the view in the articulation migration: the assets branch
on a "binding or None" pattern for optional/absent bindings (tendon types on a
tendon-less articulation, not-yet-created bindings), which the raising binding_for
can't express. try_binding_for returns None when the attribute is valid but not
available for the view's prims, while still raising UnknownAttribute for an invalid
name (a programming error, not an availability question).
…xView

read_into now reuses the float32 reinterpret of a destination buffer
across calls (keyed by buffer id, with a pointer-staleness guard), so the
wheel's object-identity read cache stays warm even when callers hand a
structured buffer each step -- they no longer need to maintain their own
reinterpret cache. get_attribute returns a typed array for attributes with
a known structured layout (transformf for poses, spatial_vectorf for
velocities, via a hand-maintained _ATTR_DTYPE map) and flat float32
otherwise.

This lets the asset data containers drop their bespoke _get_read_view
caching and read structured buffers straight through the view.
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Jun 22, 2026
OvPhysxView (and OvPhysxFrameView) live in isaaclab_ovphysx.sim.views,
which had no API-docs page. Add the automodule stub and wire it into the
isaaclab_ovphysx autosummary so the new binding-manager view shows up in
the rendered API reference alongside assets / cloner / physics.
Three fixes from the Part 1 review:

- get_attribute (no out): route the freshly allocated buffer through
  _as_binding_view directly instead of the id()-keyed read cache. A fresh
  buffer can never hit the cache and would leak one entry (keeping the
  buffer alive) per call in a step loop; the cache only pays off for a
  reused out/dst buffer. Add a regression test asserting the no-out path
  leaves _read_views empty.

- Raise a dedicated DtypeMismatch instead of ShapeMismatch when a buffer's
  scalar element type is not float32, so a dtype error no longer reads as a
  dimensions error. Update the affected tests and the Raises docstrings.

- Make the view Warp-native: drop the fragile __module__ string-match that
  auto-converted Torch tensors on writes. Callers bridge a Torch tensor
  with wp.from_torch(t), keeping the device policy explicit and avoiding an
  optional Torch dependency.
Documentation/comment clarifications from the isaac-sim#6224 review (no behavior change):

- Narrow the documented contract to float32-only: attribute_names/has_attribute
  and the module docstring now state that a listed name is name-validity, not a
  dtype-support promise; non-float dtype handling awaits wheel dtype metadata.
- Mark _READ_ONLY_NAMES explicitly temporary; name the three access modes
  (read/write, read-only, write-only) and the wheel access_mode enum that should
  replace the table.
- Document key_aliases as an internal collection adapter, not general public API,
  pending descriptor metadata.
- Make the view test scope explicit: mock API mechanics here; live
  CPU-only-on-GPU / read-only+write-only / structured read_into coverage lives in
  the asset-integration tests.
…xView

Replace the three per-attribute create_tensor_binding calls and their
manual float32 reinterpret views with a single OvPhysxView over the
rigid-parent pattern. Bindings are obtained via binding_for and read via
read_into, which fills the structured kernel buffers in place. The
CPU-only RIGID_BODY_COM_POSE keeps its pinned-host staging on a GPU sim.
No public API or numeric behavior changes.
@AntoineRichard AntoineRichard force-pushed the antoiner/feat/ovphysx_pva_view branch from 41e41c6 to f4d7c63 Compare June 23, 2026 10:01
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Jun 23, 2026
@AntoineRichard AntoineRichard marked this pull request as ready for review June 25, 2026 11:25
@greptile-apps

greptile-apps Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR is part 5 of the OVPhysX view-migration series: it introduces OvPhysxView, a string-keyed lazy binding manager over OVPhysX TensorBinding handles, and rewires the PVA sensor's _initialize_impl and _update_buffers_impl to use it in place of three direct create_tensor_binding calls.

  • OvPhysxView (ovphysx_view.py) provides get_attribute / read_into / set_attribute with device-policy enforcement, a float32-reinterpret cache for structured-dtype buffers, and a binding_for escape hatch; no public API change to the sensor layer.
  • Pva._initialize_buffers_impl removes the manual wp.array(ptr=...) reinterpret views; CPU staging for RIGID_BODY_COM_POSE is preserved using a pinned wp.transformf buffer and wp.copy, keeping numeric behavior identical.
  • A 587-line unit test suite exercises construction, get/set/read-into paths, device policy, dtype safety, and key-alias remapping against mock bindings.

Confidence Score: 3/5

The sensor migration is correct and safe to merge; the new OvPhysxView class has a subtle read-cache design issue that can silently corrupt data in a caller-misuse scenario.

The _read_view cache maps id(dst) to a float32 reinterpret but does not re-validate the cached view's shape against the current binding on a cache hit. If a structured-dtype buffer is passed to read_into for two different attributes with different flat shapes, the stale reinterpret from the first call is silently handed to the second binding's read, writing the wrong number of floats with no error. The sensor avoids this because each buffer is owned by one attribute, but the defect lives in the shared OvPhysxView class used by the asset-migration PRs in this series.

source/isaaclab_ovphysx/isaaclab_ovphysx/sim/views/ovphysx_view.py — specifically the _read_view method and the _read_views cache design.

Important Files Changed

Filename Overview
source/isaaclab_ovphysx/isaaclab_ovphysx/sim/views/ovphysx_view.py New OvPhysxView class — string-keyed binding manager with lazy creation, float32 reinterpret cache, device-policy enforcement; cache keyed only by id(dst) without per-binding shape re-validation on hit and no lifetime management.
source/isaaclab_ovphysx/isaaclab_ovphysx/sensors/pva/pva.py PVA sensor migrated from three separate create_tensor_binding calls to one OvPhysxView; CPU/GPU staging for RIGID_BODY_COM_POSE preserved correctly with structured-dtype pinned buffer and wp.copy.
source/isaaclab_ovphysx/test/sim/test_ovphysx_view.py 587-line unit test suite covering construction, get/set/read_into paths, device policy, dtype safety, and key-alias remapping — no live binding tests as documented.
source/isaaclab_ovphysx/isaaclab_ovphysx/sim/views/init.pyi Stub updated to export OvPhysxView; lazy_export() in init.py reads this stub at runtime so no separate init.py change needed.
docs/source/api/lab_ovphysx/isaaclab_ovphysx.sim.views.rst New Sphinx automodule page for isaaclab_ovphysx.sim.views; correctly hooked into index.rst.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant PVA as Pva._update_buffers_impl
    participant OPV as OvPhysxView
    participant BC as _binding cache
    participant RC as _read_view cache
    participant PX as OVPhysX TensorBinding

    PVA->>OPV: read_into(RIGID_BODY_POSE, self._transforms)
    OPV->>BC: _binding(RIGID_BODY_POSE)
    BC-->>OPV: binding
    OPV->>RC: _read_view(transforms, binding)
    RC-->>OPV: float32 reinterpret view
    OPV->>PX: binding.read(float32_view)
    PX-->>OPV: fills transforms buffer GPU

    PVA->>OPV: read_into(RIGID_BODY_VELOCITY, self._velocities)
    OPV->>BC: _binding(RIGID_BODY_VELOCITY)
    BC-->>OPV: binding
    OPV->>RC: _read_view(velocities, binding)
    RC-->>OPV: float32 reinterpret view
    OPV->>PX: binding.read(float32_view)
    PX-->>OPV: fills velocities buffer GPU

    PVA->>OPV: read_into(RIGID_BODY_COM_POSE, self._coms_read_view)
    note over OPV: CPU-only type native device is cpu
    OPV->>BC: _binding(RIGID_BODY_COM_POSE)
    BC-->>OPV: binding
    OPV->>RC: _read_view(coms_read_view, binding)
    RC-->>OPV: float32 reinterpret pinned CPU
    OPV->>PX: binding.read(float32_view)
    PX-->>OPV: fills coms_read_view CPU

    PVA->>PVA: wp.copy(coms_buffer_GPU, coms_read_view_CPU)
    note over PVA: CPU to GPU copy only on GPU sim
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant PVA as Pva._update_buffers_impl
    participant OPV as OvPhysxView
    participant BC as _binding cache
    participant RC as _read_view cache
    participant PX as OVPhysX TensorBinding

    PVA->>OPV: read_into(RIGID_BODY_POSE, self._transforms)
    OPV->>BC: _binding(RIGID_BODY_POSE)
    BC-->>OPV: binding
    OPV->>RC: _read_view(transforms, binding)
    RC-->>OPV: float32 reinterpret view
    OPV->>PX: binding.read(float32_view)
    PX-->>OPV: fills transforms buffer GPU

    PVA->>OPV: read_into(RIGID_BODY_VELOCITY, self._velocities)
    OPV->>BC: _binding(RIGID_BODY_VELOCITY)
    BC-->>OPV: binding
    OPV->>RC: _read_view(velocities, binding)
    RC-->>OPV: float32 reinterpret view
    OPV->>PX: binding.read(float32_view)
    PX-->>OPV: fills velocities buffer GPU

    PVA->>OPV: read_into(RIGID_BODY_COM_POSE, self._coms_read_view)
    note over OPV: CPU-only type native device is cpu
    OPV->>BC: _binding(RIGID_BODY_COM_POSE)
    BC-->>OPV: binding
    OPV->>RC: _read_view(coms_read_view, binding)
    RC-->>OPV: float32 reinterpret pinned CPU
    OPV->>PX: binding.read(float32_view)
    PX-->>OPV: fills coms_read_view CPU

    PVA->>PVA: wp.copy(coms_buffer_GPU, coms_read_view_CPU)
    note over PVA: CPU to GPU copy only on GPU sim
Loading

Comments Outside Diff (2)

  1. source/isaaclab_ovphysx/isaaclab_ovphysx/sim/views/ovphysx_view.py, line 822-838 (link)

    P1 Cache keyed by id(dst) alone — no shape re-validation on cache hit for structured dtypes

    The _read_view cache maps id(dst) → the float32 reinterpret produced against a specific binding. On a cache hit the cached view is returned directly, without re-checking its flat shape against the current binding.shape. If the same structured-dtype buffer is ever passed to read_into for two different attributes whose flat shapes differ (e.g. a (N,) wp.transformf buffer used first for rigid_body_pose shape (N, 7), then for rigid_body_velocity shape (N, 6)), the stale (N, 7) reinterpret is handed to the velocity binding's read, silently writing seven floats per body into a six-float-wide region and corrupting the destination buffer with no error raised. Flat float32 buffers are safe because they are never cached, so only structured-dtype destinations are affected. Keying the cache by (id(dst), binding.shape) (or adding a shape assert on hit) would close the gap.

  2. source/isaaclab_ovphysx/isaaclab_ovphysx/sim/views/ovphysx_view.py, line 474-478 (link)

    P2 _read_views cache has no eviction and does not hold a back-reference to dst

    The cache maps id(dst) → reinterpret view and is never evicted. Each reinterpret holds a raw ptr into dst's backing allocation but carries no Python reference to dst. If dst is GC'd its warp allocation may be freed; any subsequent access through the cached view would touch freed memory. The stale-pointer guard (cached.ptr == dst.ptr) only fires when a new dst arrives at that id(), not when the cached view itself is used in a later step loop. For the sensor's persistent-buffer pattern this does not manifest, but making _read_views a WeakKeyDictionary keyed by dst directly would give both correct lifetime tracking and automatic eviction.

Reviews (1): Last reviewed commit: "Merge branch 'develop' into antoiner/fea..." | Re-trigger Greptile

Add _invalidate_initialize_callback to clear _root_view on simulation stop so
a stale view (caching post-reset bindings) is not held across the reset;
_initialize_impl rebuilds a fresh OvPhysxView on the next play.
@AntoineRichard

Copy link
Copy Markdown
Collaborator Author

We dug into the read-cache "silent corruption" concern closely, including the wheel's native code, and it's a latent gap rather than a silent-corruption bug:

  • The wheel's read/write validate the tensor shape (dtype + ndim + every dim) before any copy (validateTensorShape in ovphysxTensorBinding.cpp) and return a non-SUCCESS status that the Python wrapper raises as RuntimeError (api.py:472). A cross-attribute reuse therefore raises, it does not silently corrupt.
  • It is unreachable by shipped code: each asset/sensor owns one persistent buffer per attribute (never reused across two TensorTypes), and a binding's flat shape is constant for a session — it can only change on a full sim reset, which rebuilds the view from scratch.

So we deliberately did not add a per-hit shape guard (it would add hot-path cost to defend an unreachable, already-raising case); a dtype-aware cache key is tracked as a wheel-metadata follow-up (the wheel exposes no dtype today). We did make the reset boundary explicitly tidy — every consumer now clears _root_view in _invalidate_initialize_callback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant