HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression

TL;DR

The paper tackles the large storage burden of 3D Gaussian Splatting (3DGS) by introducing Hash-grid Assisted Context (HAC), which leverages a structured binary hash grid to model the mutual information between unorganized anchors and grid features. HAC uses the interpolated hash features as context to probabilistically model anchor attributes via Gaussian distributions and to adapt quantization with a learned Adaptive Quantization Module, enabling efficient entropy coding and aggressive compression. A combination of hash-grid context, adaptive masking, and entropy-driven losses yields substantial improvements over vanilla 3DGS ( >75× size reduction) and over Scaffold-GS ( >11×), while maintaining or improving rendering fidelity across diverse datasets. The framework remains compatible with Scaffold-GS, adds relatively modest training-time overhead, and demonstrates strong potential for large-scale 3D scene compression and fast deployment on resource-limited devices.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a promising framework for novel view synthesis, boasting rapid rendering speed with high fidelity. However, the substantial Gaussians and their associated attributes necessitate effective compression techniques. Nevertheless, the sparse and unorganized nature of the point cloud of Gaussians (or anchors in our paper) presents challenges for compression. To address this, we make use of the relations between the unorganized anchors and the structured hash grid, leveraging their mutual information for context modeling, and propose a Hash-grid Assisted Context (HAC) framework for highly compact 3DGS representation. Our approach introduces a binary hash grid to establish continuous spatial consistencies, allowing us to unveil the inherent spatial relations of anchors through a carefully designed context model. To facilitate entropy coding, we utilize Gaussian distributions to accurately estimate the probability of each quantized attribute, where an adaptive quantization module is proposed to enable high-precision quantization of these attributes for improved fidelity restoration. Additionally, we incorporate an adaptive masking strategy to eliminate invalid Gaussians and anchors. Importantly, our work is the pioneer to explore context-based compression for 3DGS representation, resulting in a remarkable size reduction of over $75\times$ compared to vanilla 3DGS, while simultaneously improving fidelity, and achieving over $11\times$ size reduction over SOTA 3DGS compression approach Scaffold-GS. Our code is available here: https://github.com/YihangChen-ee/HAC

Figures (8)

• Figure 1: Top: A toy example where our method makes the size of the vanilla 3D Gaussian splitting (3DGS) model $72\times$ smaller (or $9.49\times$ smaller compared to the SoTA Scaffold-GS scaffold), with similar or better fidelity. Bottom: Most existing 3DGS compression methods concentrate solely on parameter "values" using pruning or vector quantization to reduce size, ignoring the structure relations among Gaussians. Scaffold-GS scaffold introduces anchors to cluster and neural-predict the associated Gaussians while treating each anchor point independently. Our core idea is to further exploit the inherent consistencies of anchors via a structured hash grid for a more compact 3DGS representation.
• Figure 2: Overview of our HAC framework. It is based on Scaffold-GS scaffold (top), which introduces anchors with their attributes to neural-predict 3D Gaussian attributes. Middle: Our HAC framework jointly learns structured compact hash grid (binarized for each parameter) that can be queried at any anchor location to obtain the interpolated hash feature $\bm{f}^h$. Instead of direct substitution, $\bm{f}^h$ is used as context to predict the value distributions of anchor attributes, which is essential for the subsequent entropy coding. Bottom: Our proposed context models take $\bm{f}^h$ as input and outputs ${\bm{r}}$ for the AQM (quantize anchor attribute values into a finite set) and the parameters ($\bm{\mu}$ and $\bm{\sigma}$) to model the value distributions of anchor attributes.
• Figure 3: Left chart: Statistical analysis of the value distributions of $\mathcal{A}$ on the scene "chair" of the Synthetic-NeRF dataset NeRF. All three components $\{\bm{f}^a$, $\bm{l}, \bm{o}\}$ exhibit statistical Gaussian distributions. Note that the values of $\bm{l}$ are scaled by a factor of 100 for better visualization. Right table: Experimental results of directly substituting anchor feature $\bm{f}^a$ with hash feature $\bm{f}^h$ on this dataset.
• Figure 4: RD curves for quantitative comparisons. We vary $\lambda_e$ to achieve variable bitrates. Note that ${\rm log_{10}}$ scale is used for x-axis for better visualization.
• Figure 5: Qualitative comparisons of "pompidou" from BungeeNeRF BungeeNeRF and "flowers" from Mip-NeRF360 mip360. PSNR and size results are given at lower-left.