RFS shape sensing

learning point cloud registration and computer vision. This repo uses Open3D and OpenCV2 to do point cloud registration and computer vision, and then achieve 3D point cloud reconstruction with mutiple RGB-D cameras.

to open the realsense GUI, input the command realsense-viewer in terminal.

To measure the bending angle between 2 CDM positions, first manually save the camera image from realsense-viewer, and then run getAngle.py to get the bending angle.

Note: this package is still under development. In the future, a QR code calibration will be added to the package to calibrate the camera position and orientation, and the pyrealsense2 package will be introduced to get the images automatically from the camera.

Log:

• 2023.08.02:
  • Modify getResistanceSeries.py and create getResistance.py. Currently string read from Arduino always has missing part, for example, "6.67 Ohms Angle: 73 degrees" and "e: 18416.67 Ohms Angle: 73 degrees".
  • Compare angles from getResistance.py and getAngle.py using snake_42(82)deg_Color.png. The result from getResistance.py is 31.53 degrees, and the result from getAngle.py is 39 degrees. Both the camera image and flexible sensor need to be calibrated.
• 2023.08.08:
  • Create a "realtime" folder. Achieved real time video with bending angle and resistance measurement, but the resistance seems to have time delay. (System structure, time.sleep, delay, etc)
  • Found an issue about realsense camera SDK: must disable auto white balance to keep lighting condition consistent.
• 2023.08.15:
  • Update User Interface: add 2 windows to show the curve of bending angle and resistance in real time.
  • Test system response time: the system response time at steady state is 0.7s. Checking the latency for arduino improves sampling frequency.
  • Add one combined figure to show angle and resistance in real time.

Appendix:

For reading the resistance from Arduino MEGA board, the following sample command can be used, which grants read and write permissions to all users for the corresponding port.

sudo chmod a+rw /dev/ttyACM0

HSV parameters for green background:

• H: 0-179
• S: 0-255
• V: 55-155

For blue markers:

• blue_lower = np.array([75, 55, 159], np.uint8)
• blue_upper = np.array([120, 255, 254], np.uint8)

For red markers:

• red_lower = np.array([159, 139, 0], np.uint8)
• red_upper = np.array([176, 255, 255], np.uint8)