NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance

TL;DR

NeuSG introduces a joint framework that fuses neural implicit surface reconstruction (NeuS) with 3D Gaussian Splatting to recover highly detailed, complete surfaces from multi-view imagery. By enforcing scale regularization to flatten Gaussians toward the surface and normal regularization to align Gaussian normals with NeuS-predicted normals, the method enables mutual refinement between the two representations. The approach achieves superior surface quality on Tanks and Temples against strong baselines and offers improved efficiency due to hashing and sparse optimization. This work highlights a practical path to combining dense geometric priors with implicit surface learning for fine-grained 3D reconstruction.

Abstract

Existing neural implicit surface reconstruction methods have achieved impressive performance in multi-view 3D reconstruction by leveraging explicit geometry priors such as depth maps or point clouds as regularization. However, the reconstruction results still lack fine details because of the over-smoothed depth map or sparse point cloud. In this work, we propose a neural implicit surface reconstruction pipeline with guidance from 3D Gaussian Splatting to recover highly detailed surfaces. The advantage of 3D Gaussian Splatting is that it can generate dense point clouds with detailed structure. Nonetheless, a naive adoption of 3D Gaussian Splatting can fail since the generated points are the centers of 3D Gaussians that do not necessarily lie on the surface. We thus introduce a scale regularizer to pull the centers close to the surface by enforcing the 3D Gaussians to be extremely thin. Moreover, we propose to refine the point cloud from 3D Gaussians Splatting with the normal priors from the surface predicted by neural implicit models instead of using a fixed set of points as guidance. Consequently, the quality of surface reconstruction improves from the guidance of the more accurate 3D Gaussian splatting. By jointly optimizing the 3D Gaussian Splatting and the neural implicit model, our approach benefits from both representations and generates complete surfaces with intricate details. Experiments on Tanks and Temples verify the effectiveness of our proposed method.

Figures (5)

• Figure 1: An illustration of joint optimization of implicit surface reconstruction and 3D Gaussian Splatting.
• Figure 2: The NeuSG framework includes three principal components: 1) Optimization of neural implicit surface reconstruction. 2) Geometric constraints from point clouds generated from Gaussian Splatting, as formalized in Eq. \ref{['eq:pt_loss']}. 3) Refinement of Gaussian Splatting through normal alignment, as detailed in Eq. \ref{['eq:align_normal']}.
• Figure 3: An illustration of
• Figure 4: Qualitative comparison on Tanks and Temples dataset knapitsch2017tanks. NeuSG excels in achieving both complete and intricately detailed surfaces, in contrast to baseline approaches which often result in surfaces that are either incomplete or marred by noise.
• Figure 5: The scale regularizations for Gaussian Splatting. Left: Scale regularization. Right: Normal regularization.