Opti-Acoustic Scene Reconstruction in Highly Turbid Underwater Environments

TL;DR

This work tackles underwater scene reconstruction in highly turbid water by fusing a forward-looking imaging sonar with a monocular camera. It replaces point-feature reliance with region-based optical segmentation and projects sonar beams into the image to associate range with image-based elevation, yielding fully defined 3D points in real time from a single pose. The approach shows competitive accuracy with much greater scene coverage across turbidity levels, validated in indoor tanks and real marina field tests, and is released as open-source to enable reproducibility and community adoption. The method advances manipulation-ready perception in turbid environments by leveraging opto-acoustic data fusion without training requirements, albeit with assumptions about constant range per sonar beam and overlapping sensor FOV.

Abstract

Scene reconstruction is an essential capability for underwater robots navigating in close proximity to structures. Monocular vision-based reconstruction methods are unreliable in turbid waters and lack depth scale information. Sonars are robust to turbid water and non-uniform lighting conditions, however, they have low resolution and elevation ambiguity. This work proposes a real-time opti-acoustic scene reconstruction method that is specially optimized to work in turbid water. Our strategy avoids having to identify point features in visual data and instead identifies regions of interest in the data. We then match relevant regions in the image to corresponding sonar data. A reconstruction is obtained by leveraging range data from the sonar and elevation data from the camera image. Experimental comparisons against other vision-based and sonar-based approaches at varying turbidity levels, and field tests conducted in marina environments, validate the effectiveness of the proposed approach. We have made our code open-source to facilitate reproducibility and encourage community engagement.

Figures (8)

• Figure 1: Fields of view corresponding to the hardware arrangement in (a) is shown in (b); the red swath is from the camera and the blue is from the sonar.
• Figure 2: Forward looking imaging sonar model. The point $\mathbf{P}_s$ can be represented by $[r, \theta, \phi]^T$ in a spherical coordinate frame. The range $r$ and the bearing angle $\theta$ of $\mathbf{P}_s$ are measured, while the elevation angle $\phi$ is not captured in the resulting 2D sonar image. The 3D transformation $\mathbf{T}_{s \to c}$ from sonar to camera frame is also depicted.
• Figure 3: System Architecture: An overview of the proposed pipeline.
• Figure 4: Tank pier pilings reconstruction results in different types of water. Each column in the image shows results for a specific water type. The top row shows example images of original tank water (Type I) and other emulated water types (Type 5C, 7C and 9C). The middle row shows the ground-truth CAD model in gray, with different colored point clouds depicting each algorithm outputs. This row also shows the coverage of each algorithm expressed as voxel count. The bottom row shows absolute distance error values and error value distributions.
• Figure 5: Tank sea wall reconstruction results in different types of water. Each column in the image shows results for a specific water type. The top row shows example images of original tank water (Type I) and other emulated water types (Type 5C, 7C and 9C). The middle row shows the ground-truth CAD model in gray, with different colored point clouds depicting each algorithm outputs. This row also shows the coverage of each algorithm expressed as voxel count. The bottom row shows absolute distance error values and error value distributions.