Towards Versatile Opti-Acoustic Sensor Fusion and Volumetric Mapping

Abstract

Accurate 3D volumetric mapping is critical for autonomous underwater vehicles operating in obstacle-rich environments. Vision-based perception provides high-resolution data but fails in turbid conditions, while sonar is robust to lighting and turbidity but suffers from low resolution and elevation ambiguity. This paper presents a volumetric mapping framework that fuses a stereo sonar pair with a monocular camera to enable safe navigation under varying visibility conditions. Overlapping sonar fields of view resolve elevation ambiguity, producing fully defined 3D point clouds at each time step. The framework identifies regions of interest in camera images, associates them with corresponding sonar returns, and combines sonar range with camera-derived elevation cues to generate additional 3D points. Each 3D point is assigned a confidence value reflecting its reliability. These confidence-weighted points are fused using a Gaussian Process Volumetric Mapping framework that prioritizes the most reliable measurements. Experimental comparisons with other opti-acoustic and sonar-based approaches, along with field tests in a marina environment, demonstrate the method's effectiveness in capturing complex geometries and preserving critical information for robot navigation in both clear and turbid conditions. Our code is open-source to support community adoption.

Figures (6)

• Figure 1: Overview. (a) Our BlueROV2 platform; (b) an illustration of the fields of view of its camera and sonars, with the gray volume representing the camera and the blue/red volumes indicating the horizontal/vertical sonars respectively; (c) an illustration of our proposed volumetric mapping pipeline.
• Figure 2: Forward looking imaging sonar model. The point $\mathbf{P}_s$ can be represented by $[r, \theta, \phi]^T$ in a spherical coordinate frame. The range $r$ and the bearing angle $\theta$ of $\mathbf{P}_s$ are measured, while the elevation angle $\phi$ is not captured in the resulting 2D sonar image. The 3D transformation $\mathbf{T}_{s \to c}$ from sonar to camera frame is also depicted.
• Figure 3: Testing environments and representative sonar images. Environments include: Tank Single Disk (left column), Tank Double Disk (middle column), and Marina Pier (right column). The top row shows an example image of each environment, the middle row shows an example horizontal sonar image, and the bottom row shows an example vertical sonar image.
• Figure 4: Tank single disk mapping results. Each column in the image shows the voxel map result for a different method. From left to right the order is: Octo SS RGB, GP SS RGB, GPC SS RGB (proposed approach highlighted in blue), GP S RGB, and GP SS. Voxel colors depict height.
• Figure 5: Tank double disk mapping results. Each column in the image shows the voxel map result for a different method. From left to right the order is: Octo SS RGB, GP SS RGB, GPC SS RGB (proposed approach highlighted in blue), GP S RGB, and GP SS. Voxel colors depict height.