Add General Visualizer Marker Support for Newton based Visualizers#6532
Add General Visualizer Marker Support for Newton based Visualizers#6532matthewtrepte wants to merge 3 commits into
Conversation
Greptile SummaryThis PR adds generic USD-to-Newton mesh conversion for visualization markers, replacing the previous hardcoded DexCube workaround by leveraging the new
Confidence Score: 3/5The core USD marker loading path is well-tested and robust, but the deformables demo unconditionally exposes the Newton visualizer to users without any guard, which could cause silent runtime failures in a user-facing script. The Newton marker implementation is solid and the five new tests cover the critical paths. The main concern is deformables.py: the previous explicit restriction was removed without any corresponding implementation of deformable support in Newton. The Newton RC pin also introduces a pre-release as a hard dependency. scripts/demos/deformables.py — the Newton visualizer guard was removed; confirm whether Newton now supports deformable bodies before landing this change. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["UsdFileCfg marker_cfg"] --> B["_infer_newton_marker_cfg()"]
B --> C{"usd_path ends with arrow_x.usd?"}
C -- Yes --> D["_NewtonMarkerSpec renderer=mesh, mesh_type=arrow"]
C -- No --> E{"usd_path ends with frame_prim.usd?"}
E -- Yes --> F["_NewtonMarkerSpec renderer=frame"]
E -- No --> G["_load_usd_mesh(usd_path)"]
G --> H{"nusd.get_mesh direct path"}
H -- success --> I["return newton.Mesh"]
H -- "ValueError: No prims found" --> J["Instanceable USD fallback: traverse prototypes"]
J --> K{"Mesh prim found?"}
K -- Yes --> L["nusd.get_mesh prim + bake xform"]
L --> I
K -- No --> M["log warning, return None"]
H -- "other exception" --> N["outer except, log warning, return None"]
I --> O["_NewtonMarkerSpec renderer=mesh, mesh_type=usd, preloaded_mesh"]
M --> P["_NewtonMarkerSpec renderer=none"]
N --> P
O --> Q["_ensure_mesh_registered viewer.log_mesh"]
Q --> R["viewer.log_instances"]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A["UsdFileCfg marker_cfg"] --> B["_infer_newton_marker_cfg()"]
B --> C{"usd_path ends with arrow_x.usd?"}
C -- Yes --> D["_NewtonMarkerSpec renderer=mesh, mesh_type=arrow"]
C -- No --> E{"usd_path ends with frame_prim.usd?"}
E -- Yes --> F["_NewtonMarkerSpec renderer=frame"]
E -- No --> G["_load_usd_mesh(usd_path)"]
G --> H{"nusd.get_mesh direct path"}
H -- success --> I["return newton.Mesh"]
H -- "ValueError: No prims found" --> J["Instanceable USD fallback: traverse prototypes"]
J --> K{"Mesh prim found?"}
K -- Yes --> L["nusd.get_mesh prim + bake xform"]
L --> I
K -- No --> M["log warning, return None"]
H -- "other exception" --> N["outer except, log warning, return None"]
I --> O["_NewtonMarkerSpec renderer=mesh, mesh_type=usd, preloaded_mesh"]
M --> P["_NewtonMarkerSpec renderer=none"]
N --> P
O --> Q["_ensure_mesh_registered viewer.log_mesh"]
Q --> R["viewer.log_instances"]
Reviews (1): Last reviewed commit: "add general support for viz markers on n..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| conflict_handler="resolve", | ||
| ) | ||
| parser.add_argument("--physics", default="physx", choices=["physx", "newton_mjwarp"], help="Physics backend.") | ||
| # Newton visualizer not supported for deformables | ||
| parser.add_argument("--visualizer", default="kit", choices=["kit"], help="Visualizer backend.") | ||
| add_launcher_args(parser) |
There was a problem hiding this comment.
The original code explicitly blocked the Newton visualizer for deformables with the comment "Newton visualizer not supported for deformables." This PR removes both the --visualizer argument (restricted to ["kit"]) and the parser.set_defaults(visualizer=["kit"]) call, so running ./isaaclab.sh -p scripts/demos/deformables.py --visualizer newton would now accept the Newton visualizer without any guard. If the Newton visualizer still doesn't support deformable bodies, users will get a runtime failure with no helpful error message. The PR description and the rest of the changes only address USD mesh markers, not deformable-body rendering in Newton — is this removal intentional?
|
|
||
| _NEWTON_TEST_USD = os.path.join(os.path.dirname(_newton.__file__), "tests", "assets", "cube_cylinder.usda") | ||
|
|
There was a problem hiding this comment.
_NEWTON_TEST_USD is resolved at module-import time by joining newton.__file__'s directory with "tests/assets/cube_cylinder.usda". Package builds (e.g. wheel distributions) routinely strip the tests/ directory, so this path will silently not exist in many environments. When _load_usd_mesh fails on a missing file, the marker degrades to renderer="none" and only emits a log warning — meaning the demo's "local_usd_mesh" marker disappears without any user-facing explanation.
7fe304c to
e706ea2
Compare
Replace the hardcoded per-asset USD detection (dexcube textured box, fallback-to-none for unknown paths) with a generic loader that opens any UsdFileCfg path via Usd.Stage and converts the first UsdGeom.Mesh prim to a Newton Mesh using newton.usd.get_mesh. Material properties (color, texture) are resolved automatically from the USD material. Remove _create_textured_box_mesh, _MeshData, _DEX_CUBE_TEXTURE_URL, and the "textured_box" mesh_type. Add "usd" mesh_type backed by a preloaded_mesh field on _NewtonMarkerSpec. Fix _ensure_mesh_registered to handle None normals/uvs returned by newton.Mesh.create_from_usd.
Remove --physics and --visualizer args from scripts/demos/deformables.py and scripts/demos/markers.py, restoring the standard AppLauncher pattern and simulation_app.is_running() loop. The deformables script now uses PhysX unconditionally; the Newton conditional import branches are dropped. Also trim verbose docstring paragraphs from _load_usd_mesh and remove the misleading "uniform scale" qualifier from the normals-not-transformed note.
e706ea2 to
76a4684
Compare
1 participant
Description
Previously all USD based visualizer markers were not supported by Newton based visualizers, since there wasn't support for general USD -> Newton Mesh conversion.
Newton recently added this API, so we can add general support and remove hardcoded support for specific markers, like the dexcube targets.
Note this PR requires updating Newton to 1.4.0 (which is currently breaking some tests)
Type of change
Screenshots
Please attach before and after screenshots of the change if applicable.
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there