GaussianForest: Hierarchical-Hybrid 3D Gaussian Splatting for Compressed Scene Modeling

TL;DR

The paper tackles the storage burden of 3D Gaussian Splatting by introducing GaussianForest, a hierarchical forest of hybrid Gaussians that stores explicit attributes at leaves while sharing implicit attributes in higher levels. It employs adaptive growth and pruning to allocate resources where needed and shrink everywhere else, achieving substantial compression without sacrificing rendering quality or speed. Across 21 real and synthetic scenes, GaussianForest achieves comparable or superior rendering fidelity to the state of the art while reducing model size by roughly an order of magnitude (over $10\times$), and ablations validate the benefits of the hybrid representation, forest structure, and adaptive optimization. This approach enables scalable, real-time-capable scene modeling on consumer hardware and opens avenues for unsupervised scene segmentation via implicit attribute sharing.

Abstract

The field of novel-view synthesis has recently witnessed the emergence of 3D Gaussian Splatting, which represents scenes in a point-based manner and renders through rasterization. This methodology, in contrast to Radiance Fields that rely on ray tracing, demonstrates superior rendering quality and speed. However, the explicit and unstructured nature of 3D Gaussians poses a significant storage challenge, impeding its broader application. To address this challenge, we introduce the Gaussian-Forest modeling framework, which hierarchically represents a scene as a forest of hybrid 3D Gaussians. Each hybrid Gaussian retains its unique explicit attributes while sharing implicit ones with its sibling Gaussians, thus optimizing parameterization with significantly fewer variables. Moreover, adaptive growth and pruning strategies are designed, ensuring detailed representation in complex regions and a notable reduction in the number of required Gaussians. Extensive experiments demonstrate that Gaussian-Forest not only maintains comparable speed and quality but also achieves a compression rate surpassing 10 times, marking a significant advancement in efficient scene modeling. Codes will be available at https://github.com/Xian-Bei/GaussianForest.

Figures (6)

• Figure 1: Quantitative comparison across 13 real-world scenes from three datasets on rendering quality, model size, and rendering speed. The size of each point in the figure indicates the corresponding model size (in MB). Our GaussianForest (GF) excels in adeptly balancing rendering speed and model size. Across all scenarios, GF achieves the highest speed-to-size ratio, surpassing all baselines by a large margin while ensuring high-fidelity rendering quality.
• Figure 2: Illustration of the proposed GaussianForest. GaussianForest hierarchically represents a scene as a forest composed of hybrid Gaussians, where non-leaf nodes capture their implicit attributes, while leaf nodes characterize explicit ones. Initiated from a compact set of singly linked lists via K-Means, GaussianForest adaptively grows in complex regions based on cumulative gradients to swiftly fit the scene. Leaf nodes with scaling and opacity below certain thresholds are considered trivial and subsequently removed. Such node count control ensures compact representations without compromising rendering quality while contributing to the acceleration of both training and rendering.
• Figure 3: Visualization of two Gaussian trees. A drum head is approximately modeled by a single Gaussian tree, requiring far fewer parameters compared to the thousands of explicit Gaussians in 3DGS. GaussianForest also shows inherent clustering ability, naturally segmenting similar regions without any supervisory information. During optimization, adjacent regions with similar geometric and color features tend to aggregate under the same parent node.
• Figure 4: Visualization of quantitative comparisons on Mip-NeRF360 and Tanks&Temples datasets. The horizontal and vertical axes represent rendering speed and quality, respectively. Each point's size in the figure indicates the corresponding model size in MB. This comparison serves to highlight the superiority of our approach.
• Figure 5: Qualitative comparisons illustrating rendering quality, with images generated from held-out test views.