ExGS: Extreme 3D Gaussian Compression with Diffusion Priors

TL;DR

ExGS tackles the large storage burden of 3D Gaussian Splatting by introducing a two-stage, data-driven compression framework that unifies Universal Gaussian Compression (UGC) with GaussPainter. UGC performs re-optimization-free pruning using a global significance score and voxel-based sampling, augmented by adaptive amplification to preserve structure under extreme compression. GaussPainter then uses diffusion priors with latent supervision and mask-guided conditioning to restore high-quality renderings from heavily pruned scenes in near real-time, leveraging a lightweight VAE and a one-step diffusion schedule. The approach achieves compression ratios exceeding $100\times$ (e.g., reducing a typical 354.77 MB model to about 3.31 MB) while preserving geometry and appearance, and demonstrates strong performance on indoor and outdoor datasets with real-time rendering, underscoring diffusion priors’ crucial role in bridging aggressive compression and rendering fidelity.

Abstract

Neural scene representations, such as 3D Gaussian Splatting (3DGS), have enabled high-quality neural rendering; however, their large storage and transmission costs hinder deployment in resource-constrained environments. Existing compression methods either rely on costly optimization, which is slow and scene-specific, or adopt training-free pruning and quantization, which degrade rendering quality under high compression ratios. In contrast, recent data-driven approaches provide a promising direction to overcome this trade-off, enabling efficient compression while preserving high rendering quality. We introduce ExGS, a novel feed-forward framework that unifies Universal Gaussian Compression (UGC) with GaussPainter for Extreme 3DGS compression. UGC performs re-optimization-free pruning to aggressively reduce Gaussian primitives while retaining only essential information, whereas GaussPainter leverages powerful diffusion priors with mask-guided refinement to restore high-quality renderings from heavily pruned Gaussian scenes. Unlike conventional inpainting, GaussPainter not only fills in missing regions but also enhances visible pixels, yielding substantial improvements in degraded renderings. To ensure practicality, it adopts a lightweight VAE and a one-step diffusion design, enabling real-time restoration. Our framework can even achieve over 100X compression (reducing a typical 354.77 MB model to about 3.31 MB) while preserving fidelity and significantly improving image quality under challenging conditions. These results highlight the central role of diffusion priors in bridging the gap between extreme compression and high-quality neural rendering. Our code repository will be released at: https://github.com/chenttt2001/ExGS

Figures (9)

• Figure 1: Qualitative comparison under different compression ratios. LightGS* indicates the optimized version of LightGSfan2024lightgaussian. LightGS* achieves only limited compression (around $3\times$) and produces severe visual artifacts. In contrast, our method realizes orders-of-magnitude higher compression (exceeding $100\times$ in some cases) while preserving scene geometry and appearance realism. Even at aggressive compression ratios, our reconstructions remain close to ground-truth quality, demonstrating the robustness of the proposed framework.
• Figure 2: The overall pipeline of ExGS. Dense Gaussians are compressed into a compact 3 MB representation via pruning and lightweight modules, then decompressed and processed by the VAE and UNet to reconstruct high-quality images. This design enables efficient storage, fast transmission, and high-fidelity rendering.The dashed arrows in the figure indicate components that are used only during training.
• Figure 3: Visualization of global significance scoring and voxel-based correction. (a) shows rendered views of Gaussian scenes. (b) presents heatmaps of the raw score, which emphasize textures but fail in sparse regions. (c) shows the corrected distribution after voxel-based pruning, yielding more uniform coverage and improved preservation of structure and details.
• Figure 4: Comparison of results without and with latent supervision. Latent loss effectively removes artifacts and improves reconstruction fidelity.
• Figure 5: Qualitative results at a 10% compression ratio on ScanNet++, Mip-NeRF360 and Replica. Visual comparisons show that our method reconstructs scenes with higher fidelity and completeness compared to baseline approaches. All generative methods take as input the same UGC-enhanced rendered images for fair comparison.