Bayesian uncertainty analysis for underwater 3D reconstruction with neural radiance fields

TL;DR

A spatial perturbation field D_omega based on Bayes' rays in SeaThru-NeRF is introduced and Laplace approximation is performed to obtain a Gaussian distribution N(0,Sigma) of the parameters omega, where the diagonal elements of Sigma correspond to the uncertainty at each spatial location.

Abstract

Neural radiance fields (NeRFs) are a deep learning technique that can generate novel views of 3D scenes using sparse 2D images from different viewing directions and camera poses. As an extension of conventional NeRFs in underwater environment, where light can get absorbed and scattered by water, SeaThru-NeRF was proposed to separate the clean appearance and geometric structure of underwater scene from the effects of the scattering medium. Since the quality of the appearance and structure of underwater scenes is crucial for downstream tasks such as underwater infrastructure inspection, the reliability of the 3D reconstruction model should be considered and evaluated. Nonetheless, owing to the lack of ability to quantify uncertainty in 3D reconstruction of underwater scenes under natural ambient illumination, the practical deployment of NeRFs in unmanned autonomous underwater navigation is limited. To address this issue, we introduce a spatial perturbation field D_omega based on Bayes' rays in SeaThru-NeRF and perform Laplace approximation to obtain a Gaussian distribution N(0,Sigma) of the parameters omega, where the diagonal elements of Sigma correspond to the uncertainty at each spatial location. We also employ a simple thresholding method to remove artifacts from the rendered results of underwater scenes. Numerical experiments are provided to demonstrate the effectiveness of this approach.

Figures (14)

• Figure 1: Pipeline.
• Figure 2: NeRFs architecture seathrunerf. NeRFs utilizes a MLP network, taking each 3D spatial coordinate $\mathbf{x}=(x,y,z)$ and viewing direction $\mathbf{d}=(\theta,\phi)$ in the scene as input, and outputs the corresponding color $\mathbf{c}=(r,g,b)$ and density $\sigma$.
• Figure 3: SeaThru-NeRF application scenario seathrunerf.
• Figure 4: bayes_rays considers a two-dimensional plane with a solid blue line segment and a green center line segment. Assuming two cameras capture the scene within a 60-degree cone, it can be observed that the green line segment can be replaced by many possible curves, all of which can achieve 'perfect' photometric reconstruction based on the captured pixels.
• Figure 5: Perturbation.