Compression in 3D Gaussian Splatting: A Survey of Methods, Trends, and Future Directions

TL;DR

This survey addresses the memory and storage bottlenecks of 3D Gaussian Splatting (3DGS) by systematically organizing compression methods into unstructured and structured categories. It articulates a detailed taxonomy, reviews pruning, quantization, and entropy encoding for unstructured approaches, and then covers anchor-based, context-driven, graph-based, and factorization-based structured methods, evaluating trade-offs in fidelity, memory footprint, and rendering speed. Through analysis on standard datasets (e.g., Mip-NeRF360, Tanks&Temples, Deep Blending), the work highlights that structured techniques can achieve Very high compression at the cost of throughput, while unstructured methods tend to preserve speed with competitive quality. It also discusses cross-pertilization with NeRFs and point-cloud compression to guide future directions, aiming for hybrid, hardware-aware solutions that enable real-time 3DGS on edge devices and broader applications.

Abstract

3D Gaussian Splatting (3DGS) has recently emerged as a pioneering approach in explicit scene rendering and computer graphics. Unlike traditional neural radiance field (NeRF) methods, which typically rely on implicit, coordinate-based models to map spatial coordinates to pixel values, 3DGS utilizes millions of learnable 3D Gaussians. Its differentiable rendering technique and inherent capability for explicit scene representation and manipulation positions 3DGS as a potential game-changer for the next generation of 3D reconstruction and representation technologies. This enables 3DGS to deliver real-time rendering speeds while offering unparalleled editability levels. However, despite its advantages, 3DGS suffers from substantial memory and storage requirements, posing challenges for deployment on resource-constrained devices. In this survey, we provide a comprehensive overview focusing on the scalability and compression of 3DGS. We begin with a detailed background overview of 3DGS, followed by a structured taxonomy of existing compression methods. Additionally, we analyze and compare current methods from the topological perspective, evaluating their strengths and limitations in terms of fidelity, compression ratios, and computational efficiency. Furthermore, we explore how advancements in efficient NeRF representations can inspire future developments in 3DGS optimization. Finally, we conclude with current research challenges and highlight key directions for future exploration.

Figures (10)

• Figure 1: Taxonomy of 3DGS compression methods, detailing the structured and unstructured methods.
• Figure 2: An overview of the 3DGS forward process. (a) The splatting step maps 3D Gaussians into the image plane. (b) 3DGS partitions the image into non-overlapping patches, referred to as tiles. (c) For Gaussians spanning multiple tiles, 3DGS duplicates them and assigns each a unique identifier, i.e., a tile ID. (d) Rendering the sorted Gaussians yields the pixel data for each tile. Note that the pixel and tile computation workflows are independent, enabling parallel processing. Best viewed in color. Figure inspired from DBLP:journals/corr/abs-2401-03890.
• Figure 3: A representative image from each evaluation dataset showcases the diversity of scene types. The chosen images encompass small to medium-sized natural environments from Tanks&Temples knapitsch2017tanks, Deep Blending hedman2018deep, and Mip-NeRF360 barron2022mip, ensuring a comprehensive visual comparison across different datasets.
• Figure 4: Comparison between opacity based pruning and gradient + opacity based pruning. The number of Gaussians are in millions (M).
• Figure 5: Opacity distribution before and after pruning for the truck scene. Figure taken from salman2024trimming.