High-quality Surface Reconstruction using Gaussian Surfels

TL;DR

The paper presents Gaussian surfels, a point-based surface representation that flattens 3D Gaussian ellipsoids to 2D ellipses by setting the $z$-scale to $0$, aligning the local normal with the surface to improve optimization stability. It couples differentiable Gaussian splatting with a multi-term loss, including photometric, depth-normal consistency, monocular normal priors, and opacity regularization, and adds volumetric cutting and screened Poisson meshing to produce high-quality, open surfaces from multi-view images. A self-supervised depth-normal consistency loss addresses zero-gradient issues along the normal axis, while monocular priors mitigate highlight-induced ambiguities; volumetric cutting further suppresses depth artifacts. Empirical results on DTU and BlendedMVS show competitive geometry and rendering quality with faster convergence compared to implicit methods and advantages over prior Gaussian-point methods, enabling robust and open-surface reconstructions in practice.

Abstract

We propose a novel point-based representation, Gaussian surfels, to combine the advantages of the flexible optimization procedure in 3D Gaussian points and the surface alignment property of surfels. This is achieved by directly setting the z-scale of 3D Gaussian points to 0, effectively flattening the original 3D ellipsoid into a 2D ellipse. Such a design provides clear guidance to the optimizer. By treating the local z-axis as the normal direction, it greatly improves optimization stability and surface alignment. While the derivatives to the local z-axis computed from the covariance matrix are zero in this setting, we design a self-supervised normal-depth consistency loss to remedy this issue. Monocular normal priors and foreground masks are incorporated to enhance the quality of the reconstruction, mitigating issues related to highlights and background. We propose a volumetric cutting method to aggregate the information of Gaussian surfels so as to remove erroneous points in depth maps generated by alpha blending. Finally, we apply screened Poisson reconstruction method to the fused depth maps to extract the surface mesh. Experimental results show that our method demonstrates superior performance in surface reconstruction compared to state-of-the-art neural volume rendering and point-based rendering methods.

Figures (9)

• Figure 1: The pipeline of our method.Our method involves the following steps: (a) Starting with random initialization, our method represents the surface as a set of Gaussian surfels, each with learnable position, rotation, color, opacity, and covariance; (b) Optimize the Gaussian surfels through multi-view photometric loss, depth-normal consistency loss, and normal prior loss; (c) Perform volumetric cutting on rendered depth maps, then apply Poisson meshing from rendered depth and normal to extract a high-quality mesh. Our method can automatically obtain an open surface reconstruction result.
• Figure 2: The first row: the intersection of a ray with a 3D Gaussian point is challenging to calculate precisely. As a result, methods such as SuGaR guedon2023sugar approximate the depth of intersection with the depth of the Gaussian's center point, which can introduce errors. The second row: the intersection of a ray with our Gaussian surfel can be calculated precisely, as well as its depth.
• Figure 3: Depth-normal consistency. Top: The consistency of surfel depth and normal during optimization. Bottom: Rendered depth maps with and without consistency loss, alongside the reference image.
• Figure 4: An example of error in the rendered depth. The complex distribution of Gaussian surfels after optimization makes it difficult to remove outlier Gaussian points along each ray by discarding points far from median or alpha-weighted mean.
• Figure 5: Comparisons on cutting strategies. Meshing is achieved via screened Poisson reconstruction kazhdan2013screened.