OccluGaussian: Occlusion-Aware Gaussian Splatting for Large Scene Reconstruction and Rendering

TL;DR

OccluGaussian tackles large-scale 3D Gaussian Splatting reconstruction by introducing occlusion-aware scene division and region-based rendering. It builds an attributed view graph to cluster training cameras in a way that aligns with scene layout and occlusions, enabling region-wise reconstruction with higher camera contribution. The region-based rendering culls occluded Gaussians using region-specific visibility masks, substantially speeding up rendering without sacrificing quality. Across OccluScene3D and Zip-NeRF, the method yields superior reconstruction quality and faster rendering than state-of-the-art baselines, with comprehensive ablations validating the design choices.

Abstract

In large-scale scene reconstruction using 3D Gaussian splatting, it is common to partition the scene into multiple smaller regions and reconstruct them individually. However, existing division methods are occlusion-agnostic, meaning that each region may contain areas with severe occlusions. As a result, the cameras within those regions are less correlated, leading to a low average contribution to the overall reconstruction. In this paper, we propose an occlusion-aware scene division strategy that clusters training cameras based on their positions and co-visibilities to acquire multiple regions. Cameras in such regions exhibit stronger correlations and a higher average contribution, facilitating high-quality scene reconstruction. We further propose a region-based rendering technique to accelerate large scene rendering, which culls Gaussians invisible to the region where the viewpoint is located. Such a technique significantly speeds up the rendering without compromising quality. Extensive experiments on multiple large scenes show that our method achieves superior reconstruction results with faster rendering speed compared to existing state-of-the-art approaches. Project page: https://occlugaussian.github.io.

Figures (16)

• Figure 1: (a) In large scene reconstruction from ground-level captures, there are frequent occlusions such as walls and buildings. (b) Previous scene division methods are occlusion-agnostic, so they produce regions with severe internal occlusions, leading to poor reconstruction results. (c) We propose an occlusion-aware division method to generate regions that better align with the scene layout (see the two regions highlighted in yellow in (b) and (c)), thus improving the reconstruction quality significantly. Besides, we introduce a region-based rendering technique to accelerate the rendering speed of 3D Gaussian splatting in a large scene with massive primitives.
• Figure 2: Overview of OccluGaussian.Top left: To reconstruct a large scene, we divide it into multiple regions by adopting an occlusion-aware scene division strategy. (a) We first create an attributed view graph from the posed cameras, where nodes represent cameras with positional features, and edges represent visibility correlations between them. (b) A graph clustering algorithm is applied to the view graph to cluster the cameras into multiple regions, and (c) we further refine them to obtain more balanced sizes. (d) The region boundaries are calculated based on the clustered cameras. Each region is individually reconstructed and finally merged into a complete model. Bottom left: Each region is reconstructed using three sets of training cameras: base cameras located inside the region, extended cameras providing adequate visual content of the region, and border cameras used to constrain Gaussian primitives near the boundaries. Right: We introduce a region-based rendering technique, which culls 3D Gaussians that are occluded from the region where the rendering viewpoint is located. Furthermore, we subdivide the scene into smaller sub-regions with fewer essential 3D Gaussians. This approach effectively reduces redundant computations and further boosts our rendering speed.
• Figure 3: Qualitative comparison with SOTA methods on three large scenes in the OccluScene3D dataset.
• Figure 4: Qualitative comparison with SOTA methods on three large scenes in the Zip-NeRF dataset.
• Figure 5: Different division results on the Canteen scene (upper) and Berlin scene (lower). Green lines denote the physical walls, while black lines denote the boundaries of the divided regions. Notably, CityGaussian’s division is projected onto a contracted space.