PG-NeuS: Robust and Efficient Point Guidance for Multi-View Neural Surface Reconstruction

TL;DR

PG-NeuS tackles robustness and efficiency in multi-view neural surface reconstruction by introducing probabilistic point guidance and image-space constraints. The point SDF at $x_P$ is modeled as a Gaussian with mean $f(x_P)$ and variance $\tilde{\sigma}^2(x_P)$, where $\tilde{\sigma}^2(x_P) = \sigma_0^2 + \log(1 + \exp(\sigma^2(x_P)))$, and optimized via a negative log-likelihood objective. A Neural Projection module enforces photometric consistency by projecting points onto the surface with $t_P = x_P - f(x_P) \frac{\nabla f(x_P)}{\|\nabla f(x_P)\|_2}$ and using a 4-view NCC-based loss, while a Bias Network corrects geometric bias through high-fidelity points with $f_{final}(x_P) = f(x_P) + f_b(x_P)$. The method achieves an 11x speedup over NeuS and a 33.3% accuracy improvement on DTU, with strong robustness to noisy and sparse data, yielding high-fidelity surfaces and better detail representation.

Abstract

Recently, learning multi-view neural surface reconstruction with the supervision of point clouds or depth maps has been a promising way. However, due to the underutilization of prior information, current methods still struggle with the challenges of limited accuracy and excessive time complexity. In addition, prior data perturbation is also an important but rarely considered issue. To address these challenges, we propose a novel point-guided method named PG-NeuS, which achieves accurate and efficient reconstruction while robustly coping with point noise. Specifically, aleatoric uncertainty of the point cloud is modeled to capture the distribution of noise, leading to noise robustness. Furthermore, a Neural Projection module connecting points and images is proposed to add geometric constraints to implicit surface, achieving precise point guidance. To better compensate for geometric bias between volume rendering and point modeling, high-fidelity points are filtered into a Bias Network to further improve details representation. Benefiting from the effective point guidance, even with a lightweight network, the proposed PG-NeuS achieves fast convergence with an impressive 11x speedup compared to NeuS. Extensive experiments show that our method yields high-quality surfaces with high efficiency, especially for fine-grained details and smooth regions, outperforming the state-of-the-art methods. Moreover, it exhibits strong robustness to noisy data and sparse data.

Figures (7)

• Figure 1: (a) Reconstruction results of our method with high-intensity noisy data. "Base" is the baseline for our method without point cloud inputs. "Naive PG" is a naive point guidance mechanism as explained in our Preliminaries Section. (b) Comparison with SOTA methods on accuracy and efficiency.
• Figure 2: (a) The pipeline of Point-NeuS. Our network integrates point modeling with volume rendering through the SDF network, enhancing scene representation. NeuS serves as the base framework for the volume rendering part, while the point modeling part focuses on improving the quality of surface guidance. To this end, uncertainty estimation and a Neural Projection module are designed to attenuate the effect of noise, allowing accurate reconstruction. Additionally, the Bias network further learns to compensate for the geometric bias between volume rendering and point modeling for high-fidelity data. (b) The mechanism of Neural Projection module.
• Figure 3: Convergence speed of different methods.
• Figure 4: Reconstruction results at different optimization times.
• Figure 5: Qualitative comparison of our approach with methods from different categories on DTU and BlendedMVS.