Fine-detailed Neural Indoor Scene Reconstruction using multi-level importance sampling and multi-view consistency

TL;DR

This paper proposes a novel neural implicit surface reconstruction method, named FD-NeuS, to learn fine-detailed 3D models using multi-level importance sampling strategy and multi-view consistency methodology, and introduces multi-view feature consistency and multi-view normal consistency as supervision and uncertainty respectively, which further improve the reconstruction of details.

Abstract

Recently, neural implicit 3D reconstruction in indoor scenarios has become popular due to its simplicity and impressive performance. Previous works could produce complete results leveraging monocular priors of normal or depth. However, they may suffer from over-smoothed reconstructions and long-time optimization due to unbiased sampling and inaccurate monocular priors. In this paper, we propose a novel neural implicit surface reconstruction method, named FD-NeuS, to learn fine-detailed 3D models using multi-level importance sampling strategy and multi-view consistency methodology. Specifically, we leverage segmentation priors to guide region-based ray sampling, and use piecewise exponential functions as weights to pilot 3D points sampling along the rays, ensuring more attention on important regions. In addition, we introduce multi-view feature consistency and multi-view normal consistency as supervision and uncertainty respectively, which further improve the reconstruction of details. Extensive quantitative and qualitative results show that FD-NeuS outperforms existing methods in various scenes.

Figures (3)

• Figure 1: Method overview of FD-NeuS. We utilize segmentation priors to achieve region-based ray importance sampling and use piece-wise exponential functions as weights to guide point importance sampling. Additionally, we adopt multi-view feature consistency as supervision, and use multi-view normal consistency as uncertainty to filter unreliable normal priors.
• Figure 2: Qualitative results on ScanNet dataset dai2017scannet. For each indoor scene, the first row is the top view of the whole room, and the second row is the details of the masked region. The reconstruction results of FD-NeuS visually have similar scene integrity to those of NeuRIS wang2022neuris and MonoSDF yu2022monosdf. The detailed areas are preserved better than other methods.
• Figure 3: Qualitative results of ablation study. (a) Baseline method. (b) Base model with multi-level importance sampling strategy. (c) Full model. (d) Ground truth.