POTR: Post-Training 3DGS Compression

TL;DR

POTR addresses the high storage cost of 3D Gaussian Splatting by introducing a post-training codec that forgoes fine-tuning. It combines an efficient splat-removal pruning method with a novel spherical harmonics energy compaction technique to reduce both the number of splats and the entropy of lighting coefficients, respectively, while preserving view-dependent color. The approach achieves state-of-the-art rate-distortion performance and faster inference compared to existing post-training methods, with additional gains when a simple fine-tuning step is added. The methods are designed to be parallelizable and scalable, enabling practical deployment for on-demand rendering and XR applications. Overall, POTR provides a strong, tunable alternative to training-time compression for 3DGS models, delivering smaller files and faster rendering without sacrificing quality.

Abstract

3D Gaussian Splatting (3DGS) has recently emerged as a promising contender to Neural Radiance Fields (NeRF) in 3D scene reconstruction and real-time novel view synthesis. 3DGS outperforms NeRF in training and inference speed but has substantially higher storage requirements. To remedy this downside, we propose POTR, a post-training 3DGS codec built on two novel techniques. First, POTR introduces a novel pruning approach that uses a modified 3DGS rasterizer to efficiently calculate every splat's individual removal effect simultaneously. This technique results in 2-4x fewer splats than other post-training pruning techniques and as a result also significantly accelerates inference with experiments demonstrating 1.5-2x faster inference than other compressed models. Second, we propose a novel method to recompute lighting coefficients, significantly reducing their entropy without using any form of training. Our fast and highly parallel approach especially increases AC lighting coefficient sparsity, with experiments demonstrating increases from 70% to 97%, with minimal loss in quality. Finally, we extend POTR with a simple fine-tuning scheme to further enhance pruning, inference, and rate-distortion performance. Experiments demonstrate that POTR, even without fine-tuning, consistently outperforms all other post-training compression techniques in both rate-distortion performance and inference speed.

Figures (12)

• Figure 1: Mollweide projection of the first 16 real spherical harmonics.
• Figure 2: Simplified overview of POTR. (a) Splats are removed across multiple pruning iterations based on the change in the model's mean square error (MSE) upon their removal. (b) Spherical harmonic coefficients are energy compacted, yielding a new set of lighting coefficients with a lower entropy. (c) Splat geometry is quantized and serialized using an octree, then attributes are uniformly quantized and serialized using the spatial order implied by the depth-first traversal of the octree. (d) The serialized data is entropy compressed using zstd, resulting in the final compressed bitstream.
• Figure 3: Normalized cumulative importance of the highest importance splats in the Truck model of the initial 3DGS implementation kerbl3Dgaussians. The cumulative importance is the sum of the per-splat importances $I_k$ for a given subset of splats.
• Figure 4: The mapping function $m(x)$ for $a=10$.
• Figure 5: Schematic representation of the POTR-FT encoder. $R_{\text{FT}}$ denotes the remaining number of fine-tuning cycles.