Running cuVSLAM for an AMR using RealSense D555 PoE Camera

[subbubhat99]

Hi!

Our start-up uses RealSense D555 depth cameras that are primarily PoE.

When attempting to spin up the Isaac-ROS dev container for VSLAM/visual-odometry using RealSense depth cameras with integrated IMU, on Ubuntu 22.04 and ROS2 Humble, we notice that the RealSense camera is not being detected within the container.

Running rs-dds-config returns command not found instead of listing out information about the camera. And rs-enumerate-devices also returns that device could not be detected.

I can see that the run_docker.sh bash script uses the following command to run the container:
docker run -it
--runtime=nvidia
--rm
--gpus all
--privileged
--network host
-e NVIDIA_DRIVER_CAPABILITIES=all
-e DISPLAY="$DISPLAY"
-e XAUTHORITY=$XAUTH
-e QT_X11_NO_MITSHM=1
-e _X11_NO_MITSHM=1
-e _MITSHM=0
-v $XSOCK:$XSOCK
-v $XAUTH:$XAUTH
-v .:/cuvslam
-v "$DATASETS":"$DATASETS"
$(for dev in /dev/video*; do [ -e "$dev" ] && echo "--device=$dev:$dev"; done)
-v /dev/bus/usb:/dev/bus/usb
$CUDA_MOUNT
"$IMAGE_TAG"

What changes do you think would need to be incorporated in this command or elsewhere, in order to have eth-powered depth cameras like D555 be recognized rather than their older USB-powered versions?

Would really appreciate some help here!

Kind regards,

Subbu

[zwdoescode]

Hi @subbubhat99 ! Are you using Isaac ROS 3.2? What the run_docker.sh file you are talking about? Do you mean this file https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/release-3.2/scripts/run_dev.sh?

If you are using Isaac ROS 3.2, here is a list of supported RS camera https://nvidia-isaac-ros.github.io/v/release-3.2/getting_started/hardware_setup/sensors/realsense_setup.html#camera-compatibility

[subbubhat99]

Hi @zwdoescode!

I was talking about the docker build provided at cuVSLAM/docker/run_docker.sh which I built and ran using the Dockerfile.realsense-cu12, better-suited for my system's configuration, following the given instructions.

Does this build not include isaac-ros at all?

I faced runtime errors stating "Couldn't resolve requests" when attempting to run this script ./examples/realsense/run_vio.py.

A good update I have is that after building DDS tools with -DBUILD_WITH_DDS enabled, rs-dds-config and rs-enumerate-devices returned accurate outputs and the D555 is recognized.

But I'm guessing this docker build doesn't have isaac-ros then and I need to set it up, correct values for the IMU to be the ones suitable to the D555's integrated one, change FPS from 30 to 60 for running with a resolution of 640x480, and can then expect to see VIO running?

Would love some feedback here!

Kind regards,

Subramanya

[zwdoescode]

That's correct. the cuVSLAM/docker/run_docker.sh + Dockerfile.realsense-cu12 image is a PyCuVSLAM/librealsense development image. It does not include Isaac ROS or the Isaac ROS Visual SLAM ROS 2 node.

The examples/realsense/run_vio.py script is intended to run directly through pyrealsense2 + PyCuVSLAM, so Isaac ROS is not required for that example. If you want ROS 2 integration, then you should use/setup isaac_ros_visual_slam separately (code and tutorial). Please select the proper version based on your platform.

For the D555 over Ethernet, building librealsense with -DBUILD_WITH_DDS=ON is the right direction. The current Dockerfile was written around the USB/RSUSB path and does not enable DDS by default, so it will not support the PoE/DDS path out of the box.

If you have any questions about setting up the camera with Isaac ROS, please feel free to post a question in the developer forum.