SonarSplat: Novel View Synthesis of Imaging Sonar via Gaussian Splatting

TL;DR

SonarSplat tackles realistic novel view synthesis and 3D reconstruction for imaging sonar by introducing a differentiable 3D Gaussian splatting framework tailored to sonar physics. It models per-Gaussian acoustic reflectance, azimuth streak probabilities, and elevation-ambiguous occlusions through an azimuth streak modeling module, adaptive gain, and Elevation Sampling Densification Strategy. The approach yields higher PSNR/SSIM and lower LPIPS than state-of-the-art baselines such as Neusis, Neusis-NGP, DSC, and ZSplat* on real-world datasets, and enables effective de-streaking of sonar imagery. These capabilities promise faster, more accurate sonar-based perception and 3D reconstruction, with potential extensions to data synthesis and SLAM.

Abstract

In this paper, we present SonarSplat, a novel Gaussian splatting framework for imaging sonar that demonstrates realistic novel view synthesis and models acoustic streaking phenomena. Our method represents the scene as a set of 3D Gaussians with acoustic reflectance and saturation properties. We develop a novel method to efficiently rasterize Gaussians to produce a range/azimuth image that is faithful to the acoustic image formation model of imaging sonar. In particular, we develop a novel approach to model azimuth streaking in a Gaussian splatting framework. We evaluate SonarSplat using real-world datasets of sonar images collected from an underwater robotic platform in a controlled test tank and in a real-world river environment. Compared to the state-of-the-art, SonarSplat offers improved image synthesis capabilities (+3.2 dB PSNR) and more accurate 3D reconstruction (77% lower Chamfer Distance). We also demonstrate that SonarSplat can be leveraged for azimuth streak removal.

Figures (8)

• Figure 1: We present SonarSplat, a novel 3D Gaussian splatting framework that enables novel view synthesis of realistic sonar images, image de-streaking, and 3D reconstruction. We model the acoustic reflectivity and azimuth streak probabilities for all Gaussians, yielding a 3D scene representation that can be queried for a variety of downstream tasks.
• Figure 2: Overview of SonarSplat. Our method takes as input a sensor pose and an initial set of 3D Gaussians representing the scene. Then, we transform the 3D Gaussians into the sensor's image space. We splat the reflectivity parameter $\nu_k$ to get the unsaturated image $I_u$. Additionally, we optimize and splat per-Gaussian azimuth streaking probabilities $p_k$. All of the probabilities in a range bin $r_i$ are considered in our novel Adaptive Gain Module, which adjusts the gain $A$ applied to $I_u$. Finally, we produce $\hat{I}$ by multiplying the gain $A$ by $I_u$. All parameters are optimized using gradient descent by taking losses with respect to the sonar image pixel values. All sonar images shown are polar (range/azimuth) coordinates.
• Figure 3: Illustration of the proposed Elevation Sampling Densification Strategy (ESDS). We randomly sample $N_p$ pixels (shown as green sectors) based on the magnitude of $\mathcal{L}$ then randomly place $N_g$ Gaussians (shown as green ellipses) on the elevation arc of that pixel.
• Figure 4: We present our robotic platform and selected images from our diverse set of 9 sequences used in our experiments. We show RGB images in top right corners for visualization. Our evaluation datasets focus on smaller objects and larger-scale structures such as the test-tank basin. Azimuth streaks clearly occur in the infra_360 and piling_1 sequences.
• Figure 5: We present qualitative novel view synthesis results from selected datasets and selected baselines. Note that SonarSplat produces more realistic sonar image synthesis compared to baselines and is better able to capture finer details in the environment. Note that ZSplat* indicates ZSplat trained only with the sonar loss. Images are cartesian.