Skip to content

Migrate OVPhysX FrameView + SceneDataBackend pose bindings onto OvPhysxView (view series)#6233

Open
AntoineRichard wants to merge 12 commits into
isaac-sim:developfrom
AntoineRichard:antoiner/feat/ovphysx_frameview_scenedata
Open

Migrate OVPhysX FrameView + SceneDataBackend pose bindings onto OvPhysxView (view series)#6233
AntoineRichard wants to merge 12 commits into
isaac-sim:developfrom
AntoineRichard:antoiner/feat/ovphysx_frameview_scenedata

Conversation

@AntoineRichard

Copy link
Copy Markdown
Collaborator

🔗 Stacked on top of #6224 (Part 1 — OvPhysxView)

Branched off #6224 as a sibling of the asset/sensor migrations. Targets develop; until #6224 merges the diff includes the Part 1 OvPhysxView commits. Please review #6224 first.

Description

Two more RIGID_BODY_POSE binding consumers the asset (#6225-6227) and sensor (#6228-6232) passes didn't cover, migrated together in one PR:

  • OvPhysxFrameView (sim/views/ovphysx_frame_view.py) — the frame view that cameras / XformPrim use. Builds one OvPhysxView + try_binding_for(RIGID_BODY_POSE) (preserving its explicit "matched zero bodies" error, since the view rejects a 0-count binding), and reads via read_into.
  • OvPhysxSceneDataBackend (physics/ovphysx_manager.py) — the scene-data provider. Builds one OvPhysxView per distinct rigid-body pattern, stores it on the per-pattern entry, and reads via read_into; the per-entry merge into the transform buffer is unchanged.

Both are internal refactors — no public API change (neither exposes a root_view). The ContactBinding is unrelated (separate wheel API; see #6232). The scene-data backend's bypass-init unit test hand-seeds the internal entry dict, so it now seeds a matching view stub with read_into.

Type of change

  • Internal refactor (non-breaking)

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • Changelog fragment (.skip — internal refactor, no bump)

Testing

  • test_views_xform_prim_ovphysx.py — cpu: 18 passed, cuda: 18 passed.
  • test_ovphysx_scene_data_backend.py11 passed.

OVPhysX exposes physics attributes as a loose dict of TensorType ->
TensorBinding with no view object, unlike Newton's
selection.ArticulationView and PhysX's typed tensor views. OvPhysxView
wraps the bindings for one prim pattern behind a string-keyed
get_attribute / set_attribute surface, addressing attributes by the
lowercased TensorType enum name (e.g. "articulation_dof_stiffness").
It needs no Model/State/Control source object because the TensorType
already implies where the data lives.

Prototype per docs/superpowers/specs/2026-06-17-ovphysx-view-design.md.
Adds unit tests covering name<->enum resolution, the read-only guard,
discoverability, and get/set dispatch against a fake binding (no native
simulation required).
Reworks the view from a convenience wrapper into a layer that can back the
OVPhysX asset/data classes, per the PR review of isaac-sim#6224:

- read_into(name, dst): zero-copy fill of a caller-owned, possibly
  structured-dtype buffer (e.g. wp.transformf) via a float32 reinterpret
  view -- the mechanism the data containers use today.
- set_attribute: accepts structured-dtype sources via the same reinterpret;
  non-float32-width buffers are rejected rather than silently bit-cast.
- prim_paths + key_aliases: support the fused multi-prim binding form
  (create_tensor_binding(prim_paths=[...])) and storing a binding under a
  different TensorType key, as RigidObjectCollection needs.
- binding_for(): raw TensorBinding accessor for adoption.
- _CPU_ONLY_NAMES is now derived from tensor_types._CPU_ONLY_TYPES (no drift).
- Added joint/tendon/is_fixed_base metadata passthrough; eager construction
  raises if it creates zero bindings; get_attribute allocates a fresh buffer
  (no aliasing); nested error hierarchy; PhysX/binding Protocols.

Device policy: no implicit CPU<->GPU conversion. CPU-resident property types
are read/written on CPU; a buffer on the wrong device raises DeviceMismatch
instead of being staged. Device-less host data (numpy/list) is materialized
on the binding's native device.
The OvPhysxView addition is a significant new public surface for the OVPhysX
backend, so promote the changelog fragment from a minor to a major bump.
Reword the entry to describe the binding-manager surface (read_into, the
no-device-conversion policy) and drop the internal design-note path from the
user-facing changelog.
From the second PR review of isaac-sim#6224:

- Critical: _as_binding_view now requires a float32 scalar dtype before the
  zero-copy reinterpret. A same-byte-width wrong dtype (int32) previously passed
  the count-only guard and was bit-reinterpreted into garbage on the write path;
  sub-4-byte dtypes (float16) produced a misleading "0 elements" error. Both are
  now rejected with a clear message. Regression tests added (verified failing
  without the guard).
- _resolve enforces the str | TensorType union and raises UnknownAttribute on
  anything else, instead of letting a bogus key reach the wheel.
- _binding accesses binding.count directly (a malformed binding surfaces instead
  of being masked as a phantom no-match) and surfaces the underlying
  create_tensor_binding exception in the AttributeUnavailable message.
- Added docstrings to the six metadata properties; dropped the unused
  runtime_checkable decorator.
- Tests: same-byte/sub-4-byte dtype rejection, get_attribute(out=) wrong device,
  both indices+mask forwarded, read/write through a prim_paths+key_aliases view,
  non-str/non-TensorType key, and a read-only-names-are-valid-vocabulary check.
From the API-hardening review of isaac-sim#6224. Validate at the boundary and fail loud
instead of silently corrupting, mis-binding, or no-op'ing:

- Reject non-contiguous buffers in _as_binding_view (a strided/sliced source would
  be reinterpreted as contiguous and read/write the wrong memory).
- Canonicalize the device (wp.get_device) so a "cuda" view accepts a "cuda:0"
  buffer instead of raising a spurious, unsatisfiable DeviceMismatch; falls back
  to the raw string when the device can't be resolved locally.
- Reject TensorType.INVALID via the member path too (string path already did).
- Normalize key_aliases to TensorType members so string keys are honored rather
  than silently dropped, and reject aliases that cross the CPU/GPU residency or
  read-only boundary (the device/read-only guards key on the requested type).
- Reject empty pattern/prim_paths and tensor_types-without-eager at construction.
- Eager construction with an explicit tensor_types list now surfaces a failing
  type instead of swallowing it at debug level (default sweep still skips
  inapplicable types).
- Document binding_for as an unguarded escape hatch, get_attribute's native-device
  return, and the has_attribute name-validity-vs-availability split.

Adds regression tests for each (contiguity and INVALID verified failing without
the guard).
Surfaced by dogfooding the view in the articulation migration: the assets branch
on a "binding or None" pattern for optional/absent bindings (tendon types on a
tendon-less articulation, not-yet-created bindings), which the raising binding_for
can't express. try_binding_for returns None when the attribute is valid but not
available for the view's prims, while still raising UnknownAttribute for an invalid
name (a programming error, not an availability question).
…xView

read_into now reuses the float32 reinterpret of a destination buffer
across calls (keyed by buffer id, with a pointer-staleness guard), so the
wheel's object-identity read cache stays warm even when callers hand a
structured buffer each step -- they no longer need to maintain their own
reinterpret cache. get_attribute returns a typed array for attributes with
a known structured layout (transformf for poses, spatial_vectorf for
velocities, via a hand-maintained _ATTR_DTYPE map) and flat float32
otherwise.

This lets the asset data containers drop their bespoke _get_read_view
caching and read structured buffers straight through the view.
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Jun 22, 2026
OvPhysxView (and OvPhysxFrameView) live in isaaclab_ovphysx.sim.views,
which had no API-docs page. Add the automodule stub and wire it into the
isaaclab_ovphysx autosummary so the new binding-manager view shows up in
the rendered API reference alongside assets / cloner / physics.
Three fixes from the Part 1 review:

- get_attribute (no out): route the freshly allocated buffer through
  _as_binding_view directly instead of the id()-keyed read cache. A fresh
  buffer can never hit the cache and would leak one entry (keeping the
  buffer alive) per call in a step loop; the cache only pays off for a
  reused out/dst buffer. Add a regression test asserting the no-out path
  leaves _read_views empty.

- Raise a dedicated DtypeMismatch instead of ShapeMismatch when a buffer's
  scalar element type is not float32, so a dtype error no longer reads as a
  dimensions error. Update the affected tests and the Raises docstrings.

- Make the view Warp-native: drop the fragile __module__ string-match that
  auto-converted Torch tensors on writes. Callers bridge a Torch tensor
  with wp.from_torch(t), keeping the device policy explicit and avoiding an
  optional Torch dependency.
Documentation/comment clarifications from the isaac-sim#6224 review (no behavior change):

- Narrow the documented contract to float32-only: attribute_names/has_attribute
  and the module docstring now state that a listed name is name-validity, not a
  dtype-support promise; non-float dtype handling awaits wheel dtype metadata.
- Mark _READ_ONLY_NAMES explicitly temporary; name the three access modes
  (read/write, read-only, write-only) and the wheel access_mode enum that should
  replace the table.
- Document key_aliases as an internal collection adapter, not general public API,
  pending descriptor metadata.
- Make the view test scope explicit: mock API mechanics here; live
  CPU-only-on-GPU / read-only+write-only / structured read_into coverage lives in
  the asset-integration tests.
…hrough OvPhysxView

Two more RIGID_BODY_POSE binding consumers that the asset/sensor passes
missed: OvPhysxFrameView (the frame view cameras/XformPrim use) and
OvPhysxSceneDataBackend (the scene-data provider). Both create one
RIGID_BODY_POSE tensor binding per prim pattern and read it each step.

Route both through OvPhysxView: FrameView builds one view + binding_for
(try_binding_for preserves its explicit zero-bodies error), and reads via
read_into. SceneDataBackend builds one view per distinct pattern, stores it
on the per-pattern entry, and reads via read_into; the ContactBinding-style
paths and the merged-transform logic are unchanged. Internal refactor, no
public API change; the scene-data backend's bypass-init unit test seeds a
matching view stub.

Verified: test_views_xform_prim_ovphysx.py cpu/cuda 18 passed each;
test_ovphysx_scene_data_backend.py 11 passed.
@AntoineRichard AntoineRichard force-pushed the antoiner/feat/ovphysx_frameview_scenedata branch from cd7be5d to 314d490 Compare June 23, 2026 10:02
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Jun 23, 2026
@AntoineRichard AntoineRichard marked this pull request as ready for review June 25, 2026 11:25
@greptile-apps

greptile-apps Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR introduces OvPhysxView, a string-keyed binding manager over OVPhysX TensorBinding handles, and migrates OvPhysxFrameView and OvPhysxSceneDataBackend to use it in place of direct physx.create_tensor_binding() calls. The migration is a clean internal refactor with no public API changes.

  • OvPhysxView (ovphysx_view.py): New 700-line class providing lazy binding creation, structured-dtype reinterpretation, device/dtype guards, a _read_views cache for object-identity read-cache warmth, and a try_binding_for / binding_for escape hatch for asset-internal use. A companion 587-line test suite exercises all paths against a fake PhysX.
  • OvPhysxFrameView: Replaces the direct create_tensor_binding + shape[0]==0 check with OvPhysxView + try_binding_for, and binding.read() with root_view.read_into(); the world-only (_root_view=None) path is preserved correctly.
  • OvPhysxSceneDataBackend: Replaces direct create_tensor_binding with OvPhysxView + binding_for, but the previously reachable row_count == 0 guard becomes dead code because binding_for already raises for zero-count bindings — routing the formerly-silent debug case into the except handler as a warning.

Confidence Score: 3/5

Safe to merge for OvPhysxFrameView and the new OvPhysxView itself; the scene-data backend needs one fix before merging.

The OvPhysxFrameView migration and the new OvPhysxView are correct and well-tested. The OvPhysxSceneDataBackend migration introduced a behavioural regression: binding_for raises for zero-count bindings, so the row_count == 0 guard at line 129 is unreachable and the logger.debug at line 130 is never emitted. Any USD prim that carries RigidBodyAPI but produces zero tensor-binding matches now silently triggers the except path and emits a WARNING instead of the intentional DEBUG; in scenes with partially-physics prims this would flood logs with misleading failure messages on every backend construction.

source/isaaclab_ovphysx/isaaclab_ovphysx/physics/ovphysx_manager.py — the zero-count guard is dead code and the associated debug log is unreachable.

Important Files Changed

Filename Overview
source/isaaclab_ovphysx/isaaclab_ovphysx/sim/views/ovphysx_view.py New OvPhysxView binding manager; well-structured with strong dtype/device guards, though _read_views cache keys only on id(dst) without the binding, allowing a stale shape reinterpret if the same buffer is reused across bindings with different shapes.
source/isaaclab_ovphysx/isaaclab_ovphysx/physics/ovphysx_manager.py Migrates to OvPhysxView+binding_for; the row_count == 0 guard (lines 129-131) is now unreachable dead code because binding_for already raises for zero-count bindings, turning the former silent DEBUG path for empty patterns into a spurious WARNING.
source/isaaclab_ovphysx/isaaclab_ovphysx/sim/views/ovphysx_frame_view.py Correctly migrates to OvPhysxView+try_binding_for; the None check replaces the old shape[0]==0 check idiomatically and the _root_view=None path for world-only mode is preserved.
source/isaaclab_ovphysx/test/physics/test_ovphysx_scene_data_backend.py Adds a minimal view stub with read_into to each hand-seeded entry dict; matches the new production code path correctly.
source/isaaclab_ovphysx/test/sim/test_ovphysx_view.py Comprehensive unit tests for OvPhysxView covering construction, read/write dispatch, dtype/device guards, cache behaviour, key aliases, and adversarial inputs; all against a fake PhysX backend.
source/isaaclab_ovphysx/isaaclab_ovphysx/sim/views/init.pyi Adds OvPhysxView to all and the stub import; consistent with the new module.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant FV as OvPhysxFrameView
    participant MG as OvPhysxSceneDataBackend
    participant OV as OvPhysxView
    participant PX as PhysX (wheel)
    participant BD as TensorBinding

    Note over FV,MG: __init__ / initialize_impl (once at physics ready)

    FV->>OV: OvPhysxView(physx, pattern, device)
    FV->>OV: try_binding_for(RIGID_BODY_POSE)
    OV->>PX: create_tensor_binding(tensor_type, pattern)
    PX-->>OV: "TensorBinding (count>0) or raises"
    OV-->>FV: binding or None
    Note over FV: None → raise RuntimeError (0 bodies)

    MG->>OV: OvPhysxView(physx, pattern, device)
    MG->>OV: binding_for(RIGID_BODY_POSE)
    OV->>PX: create_tensor_binding(tensor_type, pattern)
    PX-->>OV: TensorBinding
    OV-->>MG: binding (stored in entry dict)

    Note over FV,MG: Per-step reads

    FV->>OV: read_into(rigid_body_pose, pose_buf)
    OV->>OV: _read_view(dst, binding) → cached float32 reinterpret
    OV->>BD: binding.read(float32_view)
    BD-->>OV: fills pose_buf in-place

    MG->>OV: read_into(rigid_body_pose, entry.pose_buf)
    OV->>OV: _read_view(dst, binding) → cached float32 reinterpret
    OV->>BD: binding.read(float32_view)
    BD-->>OV: fills pose_buf in-place
    MG->>MG: wp.copy into merged_transforms buffer
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant FV as OvPhysxFrameView
    participant MG as OvPhysxSceneDataBackend
    participant OV as OvPhysxView
    participant PX as PhysX (wheel)
    participant BD as TensorBinding

    Note over FV,MG: __init__ / initialize_impl (once at physics ready)

    FV->>OV: OvPhysxView(physx, pattern, device)
    FV->>OV: try_binding_for(RIGID_BODY_POSE)
    OV->>PX: create_tensor_binding(tensor_type, pattern)
    PX-->>OV: "TensorBinding (count>0) or raises"
    OV-->>FV: binding or None
    Note over FV: None → raise RuntimeError (0 bodies)

    MG->>OV: OvPhysxView(physx, pattern, device)
    MG->>OV: binding_for(RIGID_BODY_POSE)
    OV->>PX: create_tensor_binding(tensor_type, pattern)
    PX-->>OV: TensorBinding
    OV-->>MG: binding (stored in entry dict)

    Note over FV,MG: Per-step reads

    FV->>OV: read_into(rigid_body_pose, pose_buf)
    OV->>OV: _read_view(dst, binding) → cached float32 reinterpret
    OV->>BD: binding.read(float32_view)
    BD-->>OV: fills pose_buf in-place

    MG->>OV: read_into(rigid_body_pose, entry.pose_buf)
    OV->>OV: _read_view(dst, binding) → cached float32 reinterpret
    OV->>BD: binding.read(float32_view)
    BD-->>OV: fills pose_buf in-place
    MG->>MG: wp.copy into merged_transforms buffer
Loading

Comments Outside Diff (2)

  1. source/isaaclab_ovphysx/isaaclab_ovphysx/physics/ovphysx_manager.py, line 128-131 (link)

    P1 Dead code + log-level regression for zero-match patterns

    view.binding_for(TT.RIGID_BODY_POSE) internally calls _binding(), which raises OvPhysxView.AttributeUnavailable whenever binding.count == 0 — so control never reaches line 128. The row_count == 0 guard is dead code and the logger.debug on line 130 is never emitted.

    The practical regression is that a pattern which discovers a prim via USD traversal but produces zero tensor-binding matches (a legitimate scenario the original author anticipated, given the explicit debug log) now falls into the except Exception handler at line 125 and emits a WARNING ("Failed to create RIGID_BODY_POSE binding …") instead. In scenes with many partially-physics prims this generates spurious warnings that would previously have been silent at the debug level.

  2. source/isaaclab_ovphysx/isaaclab_ovphysx/sim/views/ovphysx_view.py, line 805-811 (link)

    P2 _read_views cache does not key on binding — stale shape with a reused buffer

    The cache key is id(dst) alone, not (id(dst), binding). If a caller passes the same dst buffer to read_into for two different bindings with different flat shapes (e.g. rigid_body_pose followed by rigid_body_velocity on the same (N, 7) float32 buffer), the second call hits the cache, skips _as_binding_view, and hands the wheel a reinterpret view sized for the first binding's shape. The velocity binding (shape (N, 6)) would then write 6 floats per row into a 7-element view, silently corrupting the trailing element and leaving the last column stale.

    This cannot occur in the two call-sites introduced by this PR (each pose_buf is always paired with exactly one binding), but a future caller that reuses a buffer across attribute reads on the same view would get no validation error and silently wrong data.

Reviews (1): Last reviewed commit: "Merge branch 'develop' into antoiner/fea..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant