HAC++: Towards 100X Compression of 3D Gaussian Splatting

TL;DR

HAC++ addresses the heavy storage burden of 3D Gaussian Splatting by leveraging inter-anchor correlations through a structured hash grid and intra-anchor context to model anchor attribute distributions for entropy coding. Building on Scaffold-GS, it introduces a Hash-grid Assisted Context (HAC) and an intra-anchor context with a Gaussian Mixture Model, along with Adaptive Quantization and Adaptive Offset Masking to prune redundancies. The approach yields over 100x average compression and maintains or improves fidelity across diverse datasets, outperforming Scaffold-GS by substantial margins. While training time increases due to the added context modeling, HAC++ offers practical gains in storage and renders efficiently after pruning, enabling scalable deployment of 3DGS in large scenes.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a promising framework for novel view synthesis, boasting rapid rendering speed with high fidelity. However, the substantial Gaussians and their associated attributes necessitate effective compression techniques. Nevertheless, the sparse and unorganized nature of the point cloud of Gaussians (or anchors in our paper) presents challenges for compression. To achieve a compact size, we propose HAC++, which leverages the relationships between unorganized anchors and a structured hash grid, utilizing their mutual information for context modeling. Additionally, HAC++ captures intra-anchor contextual relationships to further enhance compression performance. To facilitate entropy coding, we utilize Gaussian distributions to precisely estimate the probability of each quantized attribute, where an adaptive quantization module is proposed to enable high-precision quantization of these attributes for improved fidelity restoration. Moreover, we incorporate an adaptive masking strategy to eliminate invalid Gaussians and anchors. Overall, HAC++ achieves a remarkable size reduction of over 100X compared to vanilla 3DGS when averaged on all datasets, while simultaneously improving fidelity. It also delivers more than 20X size reduction compared to Scaffold-GS. Our code is available at https://github.com/YihangChen-ee/HAC-plus.

Figures (10)

• Figure 1: Top: A toy example showcasing the effectiveness of our method, which reduces the size of the vanilla 3D Gaussian Splatting (3DGS) model by $122.5\times$ (or $16.15\times$ compared to the SoTA Scaffold-GS scaffold), with similar or better fidelity. Bottom: Most existing 3DGS compression methods typically focus on parameter "values" through pruning or vector quantization, overlooking structural relations among Gaussians. Scaffold-GS scaffold introduces anchors to cluster Gaussians and neural-predict their attributes but treats each anchor independently. In contrast, our method leverages the inherent consistencies among anchors via a structured hash grid, enabling a significantly more compressed 3DGS representation.
• Figure 2: Overview of our HAC++ framework. Built upon Scaffold-GS scaffold (left), which introduces anchors and their attributes to neural-predict 3D Gaussian attributes, HAC++ enhances compression performance by modeling both inter- and intra-anchor relations. Right: HAC++ consists of a Hash-grid Assisted Context (HAC) and an Intra-Anchor Context. HAC learns a structured and compact hash grid (binarized for each parameter), which is queried at anchor locations to generate interpolated hash features $\bm{f}^h$. Instead of directly replacing anchor features, $\bm{f}^h$ serves as context for predicting the value distributions of anchor attributes, which is crucial for entropy coding. The intra-anchor context further improves prediction accuracy by eliminating internal redundancies of anchors. Additionally, HAC outputs ${\bm{r}}$ for the Adaptive Quantization Module (AQM), which quantizes anchor attribute values into a finite set for entropy coding. An Adaptive Offset Masking strategy (middle) is integrated to prune redundant Gaussians and anchors, enhancing the model’s pruning efficiency across different rate points.
• Figure 3: Statistical analysis of the value distributions of $\mathcal{A}$ on the scene "chair" of the Synthetic-NeRF dataset NeRF. All three components $\{\bm{f}^a$, $\bm{l}, \bm{o}\}$ exhibit statistical Gaussian distributions. Note that the values of $\bm{l}$ are scaled by a factor of 100 for better visualization.
• Figure 4: Detailed training process of HAC++. We use red lines and blue lines to indicate the training process of our model and Scaffold-GS, respectively.
• Figure 5: RD curves for quantitative comparisons. We vary $\lambda$ to achieve variable bitrates. Note that ${\rm log_{10}}$ scale is used for x-axis for better visualization.