3D Gaussian Splatting with Normal Information for Mesh Extraction and Improved Rendering

TL;DR

This work tackles geometry inaccuracies in differentiable 3D Gaussian Splatting caused by photometric losses by introducing a normal-supervised regularizer that aligns the gradient of an estimated SDF, derived from Gaussians, with monocular normals from a pretrained network. The method augments the Gaussian Splatting pipeline with a gradient regularization term R(f̂) that encourages smooth, surface-aligned geometry, and extracts meshes via Poisson reconstruction on the SDF level set. It demonstrates improved mesh quality and competitive rendering realism across datasets like Mip-NeRF360, Tanks & Temples, and Deep-Blending, while maintaining scalability and efficiency. The approach offers a practical path to high-fidelity 3D surfaces suitable for downstream tasks in AR/VR, animation, and gaming, without heavily hampering rendering speed.

Abstract

Differentiable 3D Gaussian splatting has emerged as an efficient and flexible rendering technique for representing complex scenes from a collection of 2D views and enabling high-quality real-time novel-view synthesis. However, its reliance on photometric losses can lead to imprecisely reconstructed geometry and extracted meshes, especially in regions with high curvature or fine detail. We propose a novel regularization method using the gradients of a signed distance function estimated from the Gaussians, to improve the quality of rendering while also extracting a surface mesh. The regularizing normal supervision facilitates better rendering and mesh reconstruction, which is crucial for downstream applications in video generation, animation, AR-VR and gaming. We demonstrate the effectiveness of our approach on datasets such as Mip-NeRF360, Tanks and Temples, and Deep-Blending. Our method scores higher on photorealism metrics compared to other mesh extracting rendering methods without compromising mesh quality.

Figures (5)

• Figure 1: Normals before (a) and after (b) optimization with our regularizer. Also depicted are ground truth normals (c) and the final rendered image (d). Constraining the SDF gradient to be the normal enables smoother geometric transitions in GS.
• Figure 2: Rendered images using our regularization.
• Figure 3: UV-Textured Meshes extracted after optimizing with our regularizing term, with colors rasterized onto the mesh.
• Figure 4: Textureless meshes extracted from Gaussians (a) for vanilla GS and (b) after optimizing with our regularizing term for the scene in \ref{['fig:f9']}. The 3DGS mesh contains holes, and is very noisy.
• Figure 5: Computed normals without and with regularization.