GaussianSpa: An "Optimizing-Sparsifying" Simplification Framework for Compact and High-Quality 3D Gaussian Splatting

TL;DR

GaussianSpa addresses the memory bottleneck of 3D Gaussian Splatting by casting simplification as a constrained optimization that enforces sparsity gradually during training. The core idea, an optimizing-sparsifying scheme built on an augmented Lagrangian, alternates between optimizing Gaussian attributes and projecting onto a sparsity-constrained space, removing near-zero Gaussians only after convergence. Empirically, GaussianSpa delivers higher rendering quality with far fewer Gaussians than prior methods, e.g., about a 0.9 dB PSNR gain on Deep Blending with 10× fewer Gaussians, and up to 0.4–0.7 dB gains on other benchmarks, while enabling adaptive density distribution that preserves high-frequency details. The approach offers a general, compression-friendly pathway for compact, high-fidelity 3D representations in view synthesis, with potential to synergize with SH/quantization-based storage reductions and other optimization techniques.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a mainstream for novel view synthesis, leveraging continuous aggregations of Gaussian functions to model scene geometry. However, 3DGS suffers from substantial memory requirements to store the multitude of Gaussians, hindering its practicality. To address this challenge, we introduce GaussianSpa, an optimization-based simplification framework for compact and high-quality 3DGS. Specifically, we formulate the simplification as an optimization problem associated with the 3DGS training. Correspondingly, we propose an efficient "optimizing-sparsifying" solution that alternately solves two independent sub-problems, gradually imposing strong sparsity onto the Gaussians in the training process. Our comprehensive evaluations on various datasets show the superiority of GaussianSpa over existing state-of-the-art approaches. Notably, GaussianSpa achieves an average PSNR improvement of 0.9 dB on the real-world Deep Blending dataset with 10$\times$ fewer Gaussians compared to the vanilla 3DGS. Our project page is available at https://noodle-lab.github.io/gaussianspa/.

Figures (14)

• Figure 1: We present GaussianSpa, enabling high-quality and compact view synthesis with superior rendering of details. Compared to the existing state-of-the-art method, Mini-Splatting fang2024mini, our GaussianSpa captures detail-rich textures and low-frequency background more accurately with fewer Gaussians.
• Figure 2: PSNR curves of hand-crafted criteria-based pruning methods. Gaussians are removed by 85% at iteration 25K.
• Figure 3: Overall workflow of our proposed GaussianSpa framework.
• Figure 4: Visual quality results on the Drjohnson and Counter scenes, compared to existing simplification approaches and vanilla 3DGS. The numbers of remaining Gaussians are displayed. It is observed that our GaussianSpa recovers details closest to the ground truth in the actual rendering outcomes with a significantly reduced number of Gaussians.
• Figure 5: Evolution of opacity distribution and PSNR for 3DGS and GaussianSpa. The "optimizing-sparsifying" starts at iteration 15K. At iteration 25K, GaussianSpa removes "zero" Gaussians, followed by a light tuning.