Three-dimensional Morphological Reconstruction of Millimeter-Scale Soft Continuum Robots based on Dual-Stereo-Vision

TL;DR

This work tackles the challenge of obtaining high-fidelity 3D morphology for millimeter-scale soft continuum robots, where traditional sensing is impractical. It introduces a dual stereo vision pipeline that captures raw point clouds from two fixed cameras and relocates them to a coherent morphology using a predefined geometric model with KD-tree nearest-neighbor matching, followed by ICP refinement and Poisson surface reconstruction. The approach demonstrates the ability to reveal morphological details on a 3.5 mm NTCR, achieving 14 of 16 notch features and providing quantitative improvements in point-density distribution. The method enables self-modeling and detailed morphological understanding for tiny soft robots, with potential extensions to other small objects and enhanced Sim2Real data generation.

Abstract

Continuum robots can be miniaturized to just a few millimeters in diameter. Among these, notched tubular continuum robots (NTCR) show great potential in many delicate applications. Existing works in robotic modeling focus on kinematics and dynamics but still face challenges in reproducing the robot's morphology -- a significant factor that can expand the research landscape of continuum robots, especially for those with asymmetric continuum structures. This paper proposes a dual stereo vision-based method for the three-dimensional morphological reconstruction of millimeter-scale NTCRs. The method employs two oppositely located stationary binocular cameras to capture the point cloud of the NTCR, then utilizes predefined geometry as a reference for the KD tree method to relocate the capture point clouds, resulting in a morphologically correct NTCR despite the low-quality raw point cloud collection. The method has been proved feasible for an NTCR with a 3.5 mm diameter, capturing 14 out of 16 notch features, with the measurements generally centered around the standard of 1.5 mm, demonstrating the capability of revealing morphological details. Our proposed method paves the way for 3D morphological reconstruction of millimeter-scale soft robots for further self-modeling study.

Figures (7)

• Figure 1: Overview of the 3D-morphological reconstruction and optimization pipeline for millimeter-scale soft robots. It can be seen that the preliminary reconstructed robot demonstrates two flat faces of point clouds due to the low resolution of stereo camera systems. After applying our method, a refined soft robot with more details and patterns can be obtained.
• Figure 2: The experimental setup with the sliding track system and dual Intel RealSense D405 cameras.
• Figure 3: Comparison of Point clouds (with RGB information) captured by Intel RealSense (a) D405 and (b) D435i.
• Figure 4: Preliminary 3D reconstruction results showing the detailed geometry of the NTCR.
• Figure 5: Optimized 3D model of the NTCR after KD-Tree-based alignment and refinement. It can be seen that, compared to the preliminary result, the optimized model has no flat surfaces due to the re-alignment of the point clouds. The notch patterns are clearer.