UW-SDF: Exploiting Hybrid Geometric Priors for Neural SDF Reconstruction from Underwater Multi-view Monocular Images

TL;DR

UW-SDF, a framework for reconstructing target objects from multi-view underwater images based on neural SDF, is proposed, and a novel few-shot multi-view target segmentation strategy using the general-purpose segmentation model (SAM) is proposed, enabling rapid automatic segmentation of unseen objects.

Abstract

Due to the unique characteristics of underwater environments, accurate 3D reconstruction of underwater objects poses a challenging problem in tasks such as underwater exploration and mapping. Traditional methods that rely on multiple sensor data for 3D reconstruction are time-consuming and face challenges in data acquisition in underwater scenarios. We propose UW-SDF, a framework for reconstructing target objects from multi-view underwater images based on neural SDF. We introduce hybrid geometric priors to optimize the reconstruction process, markedly enhancing the quality and efficiency of neural SDF reconstruction. Additionally, to address the challenge of segmentation consistency in multi-view images, we propose a novel few-shot multi-view target segmentation strategy using the general-purpose segmentation model (SAM), enabling rapid automatic segmentation of unseen objects. Through extensive qualitative and quantitative experiments on diverse datasets, we demonstrate that our proposed method outperforms the traditional underwater 3D reconstruction method and other neural rendering approaches in the field of underwater 3D reconstruction.

Figures (9)

• Figure 1: Underwater object reconstruction. Given multi-view underwater images taken by an optical camera mounted on an underwater vehicle, our UW-SDF reconstructs the target object leveraging neural SDF with hybrid 2D and 3D geometric priors.
• Figure 2: An overview of our pipeline. The captured underwater multi-view images are first enhanced (A), followed by training a neural SDF field for surface reconstruction (B). We utilize 2D and 3D hybrid geometric priors to optimize the reconstruction (C). A few-shot automatic segmentation strategy is introduced to obtain the desired foreground masks efficiently (D).
• Figure 3: Few-shot multi-view automatic target segmentation strategy. The process begins by selecting key images $I'_1...I'_k$ from the captured multi-view images $I_1...I_n$ and grouping them into $G_1...G_k$. Subsequently, few-shot annotations of masks $M'_1...M'_k$ are labeled according to the key images within each group. Finally, the SAM model is employed to perform multi-view automatic target segmentation on all the images.
• Figure 4: Positive-negative location prompts. For each group of images to be segmented, we compute local confidence maps $S^i$ for the test images $I$ based on the key viewpoint image $I_R$ and reference mask $M_R$. From these maps, two points are selected based on their highest and lowest confidence levels.
• Figure 5: Qualitative comparisons of the geometric reconstruction with state-of-the-art methods. From top to bottom are the reconstruction results for the simulation dataset DTU-Water, the real-world dataset UW-3D, and DRUVA. Zoom in for details.