PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction

TL;DR

PGSR tackles the challenge of achieving high-fidelity surface reconstruction from multi-view RGB images without geometric priors by introducing planar-based Gaussian Splatting. It flattens 3D Gaussians into 2D planes to render unbiased per-pixel plane distance and normals, then applies single-view and multi-view geometric regularizations plus exposure compensation to enforce global geometric consistency while preserving fast rendering. The approach delivers state-of-the-art geometric reconstruction on DTU, Tanks and Temples, and MiP-NeRF360 datasets with training times around an hour on a single RTX 4090, outperforming NeRF-based and prior GS-based methods in geometry and maintaining fast rendering. This work advances practical neural surface reconstruction for AR/VR and robotics by combining efficient planar Gaussians with robust geometric constraints for reliable, high-quality surfaces.

Abstract

Recently, 3D Gaussian Splatting (3DGS) has attracted widespread attention due to its high-quality rendering, and ultra-fast training and rendering speed. However, due to the unstructured and irregular nature of Gaussian point clouds, it is difficult to guarantee geometric reconstruction accuracy and multi-view consistency simply by relying on image reconstruction loss. Although many studies on surface reconstruction based on 3DGS have emerged recently, the quality of their meshes is generally unsatisfactory. To address this problem, we propose a fast planar-based Gaussian splatting reconstruction representation (PGSR) to achieve high-fidelity surface reconstruction while ensuring high-quality rendering. Specifically, we first introduce an unbiased depth rendering method, which directly renders the distance from the camera origin to the Gaussian plane and the corresponding normal map based on the Gaussian distribution of the point cloud, and divides the two to obtain the unbiased depth. We then introduce single-view geometric, multi-view photometric, and geometric regularization to preserve global geometric accuracy. We also propose a camera exposure compensation model to cope with scenes with large illumination variations. Experiments on indoor and outdoor scenes show that our method achieves fast training and rendering while maintaining high-fidelity rendering and geometric reconstruction, outperforming 3DGS-based and NeRF-based methods.

Figures (21)

• Figure 1: PGSR representation. We present a Planar-based Gaussian Splatting Reconstruction representation for efficient and high-fidelity surface reconstruction from multi-view RGB images without any geometric prior (depth or normal from pre-trained model). The courthouse reconstructed by our method demonstrates that PGSR can recover geometric details, such as textual details on the building. From left to right: input SfM points, planar-based Gaussian ellipsoid, rendered view, textured mesh, surface, and normal.
• Figure 2: Unbiased depth rendering. (a) Illustration of the rendered depth: We take a single Gaussian, flatten it into a plane, and fit it onto the surface as an example. Our rendered depth is the intersection point of rays and surfaces, matching the actual surface. In contrast, the depth from previous methods jiang2023gaussianshadercheng2024gaussianpro corresponds to a curved surface and may deviate from the actual surface. (b) We use true depth to supervise two different depth rendering methods. After optimization, we map the positions of all Gaussian points. Gaussians of our method fit well onto the actual surface, while the previous method results in noise and poor adherence to the surface.
• Figure 3: Rendered Depth. The original depth in 3DGS exhibits significant noise, while our depth is smoother and more accurate.
• Figure 4: PGSR Overview. We compress Gaussians into flat planes and render distance and normal maps, which are then transformed into unbiased depth maps. Single-view and multi-view geometric regularization ensure high precision in global geometry. Exposure compensation RGB loss enhances reconstruction accuracy.
• Figure 5: The rendering and mesh reconstruction results in various indoor and outdoor scenes that we have achieved. PGSR achieves high-precision geometric reconstruction from a series of RGB images without requiring any prior knowledge.