CompGS: Smaller and Faster Gaussian Splatting with Vector Quantization

TL;DR

CompGS addresses the large storage demands of 3D Gaussian Splatting by applying vector quantization to Gaussian parameters during training, building compact per-parameter codebooks and per-Gaussian indices, and using run-length encoding to compress the index stream. An opacity regularizer promotes pruning of near-invisible Gaussians, enabling substantial reductions in memory (40–50x) and faster rendering (2–3x) with minimal quality loss, while BitQ further increases compression to ~65x. The method preserves 3DGS's rendering speed and quality on multiple benchmarks, including AR/VR-scale datasets, making radiance-field representations more practical for edge devices. The work also provides extensive ablations and analysis on codebook design, parameter quantization, and cross-scene generalization of the codebooks.

Abstract

3D Gaussian Splatting (3DGS) is a new method for modeling and rendering 3D radiance fields that achieves much faster learning and rendering time compared to SOTA NeRF methods. However, it comes with a drawback in the much larger storage demand compared to NeRF methods since it needs to store the parameters for several 3D Gaussians. We notice that many Gaussians may share similar parameters, so we introduce a simple vector quantization method based on K-means to quantize the Gaussian parameters while optimizing them. Then, we store the small codebook along with the index of the code for each Gaussian. We compress the indices further by sorting them and using a method similar to run-length encoding. Moreover, we use a simple regularizer to encourage zero opacity (invisible Gaussians) to reduce the storage and rendering time by a large factor through reducing the number of Gaussians. We do extensive experiments on standard benchmarks as well as an existing 3D dataset that is an order of magnitude larger than the standard benchmarks used in this field. We show that our simple yet effective method can reduce the storage cost for 3DGS by 40 to 50x and rendering time by 2 to 3x with a very small drop in the quality of rendered images.

Figures (10)

• Figure 1: Inference speed vs. memory comparison. All methods except INGP achieve comparable PSNR that are reported in Table \ref{['tab:comp_sota']}. CompGS, our compressed version of 3DGS, maintains the speed and performance of 3DGS while reducing its size to the levels of NeRF based approaches. We achieve around $45\times$ compression and $2.5\times$ inference speed up with little drop in performance (CompGS-32K). A bit quantized version of this (Ours-BitQ) compresses it further to a total compression of $65\times$ with hardly noticeable difference in quality.
• Figure 2: Overview of CompGS vector quantization: We compress 3DGS using vector quantization of the parameters of the Gaussians. The quantization is performed along with the training of the Gaussian parameters. Considering each Gaussian as a vector, we perform K-means clustering to represent the $N$ Gaussians in the model with $k$ cluster centers (codes). Each Gaussian is then replaced by its corresponding code for rendering and loss calculation. The gradients wrt centers are copied to all the elements in the corresponding cluster and the non-quantized versions of the parameters are updated. Only the codebook and code assignments for each Gaussian are stored and used for inference. To further reduce the storage and inference time, we regularize opacity in the loss to encourage fully transparent Gaussians. CompGS maintains the real-time rendering property of 3DGS while compressing it by an order of magnitude.
• Figure 3: Qualitative comparison of novel view synthesis approaches. We visualize images from different scenes across datasets for SOTA NeRF, 3DGS, our CompGS and the No-SH variant of 3DGS . All methods based on 3DGS have better reconstruction of finer details like spokes of the bicycle wheel compared to NeRF approaches. Both compressed versions CompGS and 3DGS-No-SH are similar in appearance to 3DGS with no additional visually apparent errors.
• Figure 4: Effect of codebook length. We vary codebook size while quantizing one of the paramteres (SH, DC, Rotation, Scale).
• Figure 5: Effect of shared codebook. A frozen codebook trained on one scene ('Counter' scene) generalizes well to all other scenes in MipNerf-360 dataset. Only code assignments are learnt during training.