Importing Habitat Matterport 3D scenes into Isaac Sim

[prdrj]

Hello,

I am trying to load Habitat Matterport 3D scenes into Isaac Sim. The scene loads but the semantics and color are lost. Is there a proper way to convert these glb scenes into Isaac Sim compatible USD files?

Isaac Sim:

Dollhouse view from the habitat website:

[michaellin6]

Hi @prdrj thank you for posting the issue. This is a known two-part issue with HM3D GLB imports. Here's what you can try:

Textures/Colors
The converter doesn't reliably resolve embedded textures in GLB files. The workaround is to go through Blender:

Open the .glb in Blender and save the .blend file
File → External Data → Unpack Resources (extracts embedded textures to disk)
Export as OBJ with Materials → Path Mode set to Copy
Run convert_mesh.py on the resulting OBJ — it handles OBJ+MTL reliably:

bash./isaaclab.sh -p scripts/tools/convert_mesh.py
scene.obj scene.usd
--make-instanceable
--collision-approximation none
If you're on IsaacSim 5.1+ / IsaacLab 2.3+, the GLB importer is improved and may work without this workaround.

Semantics
HM3D semantic labels live in a separate annotation file and don't transfer automatically during mesh conversion. You need to apply them post-conversion using the add_labels API:
pythonimport omni.usd
from isaacsim.core.utils.semantics import add_labels

stage = omni.usd.get_context().get_stage()

for prim in stage.Traverse():
prim_name = prim.GetName()
if prim_name in your_annotation_dict:
add_labels(prim, labels=[your_annotation_dict[prim_name]], instance_name="class")
Build your_annotation_dict by parsing HM3D's hab_mp3d_category_mappings.tsv or the .semantic.glb annotations. You can also verify labels interactively via Tools → Replicator → Semantics Schema Editor before scripting at scale.
Hope this helps — related discussion: IsaacLab #4031

[prdrj]

Thank You!

Not the perfect result, but good enough.
Also, I had the habitat version of the hm3d dataset which is not compatible with blender, so I had to download the standard glb version.