Acoustic Neural 3D Reconstruction Under Pose Drift

TL;DR

This work tackles 3D reconstruction from imaging sonar under pose drift by jointly optimizing a neural implicit surface and drifting sonar poses. It introduces a differentiable acoustic rendering framework that represents geometry with a neural SDF N and radiance with M, while treating poses as learnable SE(3) parameters and backpropagating through the renderer. The objective combines an intensity loss, an Eikonal regularization, and an opacity penalty, summarized as $\mathcal{L} = \mathcal{L}_{\text{int}} + \lambda_{\text{eik}} \mathcal{L}_{\text{eik}} + \lambda_{\text{reg}} \mathcal{L}_{\text{reg}}$, optimized over $\Theta$, $\Phi$, and $\mathcal{T}$ with ADAM; surfaces are extracted via Marching Cubes. Experiments on simulated and real data show robust 3D reconstructions under significant pose drift, with results approaching drift-free baselines under moderate drift and outperforming drifting baselines in challenging scenarios.

Abstract

We consider the problem of optimizing neural implicit surfaces for 3D reconstruction using acoustic images collected with drifting sensor poses. The accuracy of current state-of-the-art 3D acoustic modeling algorithms is highly dependent on accurate pose estimation; small errors in sensor pose can lead to severe reconstruction artifacts. In this paper, we propose an algorithm that jointly optimizes the neural scene representation and sonar poses. Our algorithm does so by parameterizing the 6DoF poses as learnable parameters and backpropagating gradients through the neural renderer and implicit representation. We validated our algorithm on both real and simulated datasets. It produces high-fidelity 3D reconstructions even under significant pose drift.

Figures (8)

• Figure 1: The pipeline of our proposed method. Our approach jointly optimizes sonar neural implicit surface networks and pose parameters by minimizing the total reconstruction loss. It takes 3D samples and viewing directions—both dependent on pose estimates—as inputs and outputs the signed distance function (SDF) $\mathbf{N}$ and outgoing acoustic radiance $\mathbf{M}$ for sonar image rendering. This pipeline enables training with sonar images and odometry that may be subject to drift.
• Figure 2: Left: Simulated robot in HoloOcean HoloOceanDocs with the DVL and an example sonar frame visualized. Right: Real HAUV with the DVL and sonar frames overlaid. Note that the DVL frame is similarly oriented in both the simulated and real setups.
• Figure 3: 3D reconstruction results for (a) the $14^\circ$ and (b) the $28^\circ$ elevation aperture sonar datasets collected using the HoloOcean underwater simulator. From left to right, the images show ground-truth meshes of six different objects, followed by reconstructions from NeuSIS with ground-truth odometry, NeuSIS with drifting poses, and our proposed method. Our approach effectively restores dense 3D reconstructions despite drifting odometry, achieving results comparable to NeuSIS with ground-truth trajectories.
• Figure 4: Real-world experimental setup.
• Figure 5: Top-down view illustrating an example of a drifting DVL trajectory after noise injection (blue) alongside its corresponding DVL poses with no added noise (red) for the $14^\circ$ and $28^\circ$ elevation aperture real datasets, shown on the left and right, respectively. Noise is injected into the $x$, $y$, and $\psi$ relative poses with $\varepsilon^x, \varepsilon^y \sim \mathcal{N}(0, 0.015\ \text{m})$ and $\varepsilon^\psi \sim \mathcal{N}(0, 0.015\ \text{rad})$.