Enhancing Surface Neural Implicits with Curvature-Guided Sampling and Uncertainty-Augmented Representations

TL;DR

This paper addresses robust 3D surface reconstruction from depth images using neural implicit representations. It introduces curvature-guided sampling and an uncertainty-augmented $SDF$ framework built on a lightweight coarse voxel grid derived from depth, enabling both on-surface and off-surface sampling via first-order Taylor approximations. The method computes mean curvature directly from depth, stores gradients and uncertainty, and trains an uncertainty-aware implicit function that can reconstruct open surfaces; it also integrates with existing methods like IGR and NeuralPull, achieving state-of-the-art results on synthetic and real datasets. The practical impact lies in robust, depth-based reconstruction that handles sparse inputs and open surfaces with improved efficiency and compatibility for real-world sensing pipelines.

Abstract

Neural implicit representations have become a popular choice for modeling surfaces due to their adaptability in resolution and support for complex topology. While previous works have achieved impressive reconstruction quality by training on ground truth point clouds or meshes, they often do not discuss the data acquisition and ignore the effect of input quality and sampling methods during reconstruction. In this paper, we introduce a method that directly digests depth images for the task of high-fidelity 3D reconstruction. To this end, a simple sampling strategy is proposed to generate highly effective training data, by incorporating differentiable geometric features computed directly based on the input depth images with only marginal computational cost. Due to its simplicity, our sampling strategy can be easily incorporated into diverse popular methods, allowing their training process to be more stable and efficient. Despite its simplicity, our method outperforms a range of both classical and learning-based baselines and demonstrates state-of-the-art results in both synthetic and real-world datasets.

Figures (18)

• Figure 1: Our Reconstruction of a room (middle) with sparse input contains only $~6k$ points (left) and the estimated uncertainty (right).
• Figure 2: The summary of our pipeline. The red and blue correspond to proposed theoretical and architectural improvement. After a customized coarse voxel initialization with uncertainty $w^v$, curvature $H$, and normal $\hat{\mathbf{g}}^v$, we use curvature-guided sample on the extracted point cloud and using voxel-based sampling to generate more training points in the space.
• Figure 3: (a) The visualization of Gaussian curvatures and mean curvatures of each point. The red color indicates a high curvature area, and the blue color indicates a low curvature area. A positive mean curvature ($H > 0$) signifies a convex surface, and a negative mean curvature ($H < 0$) indicates a concave surface. Positive Gaussian curvature ($K > 0$) indicates that the surface is locally like a dome, and negative Gaussian curvature ($K < 0$) indicates that the surface is locally saddle-shaped. (b) Points gathering on high curvature effect (left) and our sampling results after considering points mean curvature (right). (c) Illustration of \ref{['eq:sdf_sample']}. For each query point that falls in voxel $\mathbf{v}$, the SDF value is the SDF value of the voxel $\psi^v$ plus the projected distance of the voxel center to the query point to the gradient direction.
• Figure 4: surface extraction with uncertainty. Black vertices represent zero uncertainty points. Red and blue vertex mean points with negative and positive SDF values, respectively.
• Figure 5: Visualization of reconstructed meshes (first three columns) of Armadillo (more visualizations cf. supplementary). The right side is the visualization of the zoomed-in part in left meshes, the initial discrete SDF stored in $64^3$ resolution cube, and continuous SDF represented by a neural network.

Theorems & Definitions (1)

• proof