Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MEGS$^{2}$: Memory-Efficient Gaussian Splatting via Spherical Gaussians and Unified Pruning

Jiarui Chen, Yikeng Chen, Yingshuang Zou, Ye Huang, Peng Wang, Yuan Liu, Yujing Sun, Wenping Wang

TL;DR

MEGS is introduced, a novel memory-efficient framework that tackles the challenge of memory compression in 3D Gaussian Splatting by jointly optimizing two key factors: the total primitive number and the parameters per primitive, achieving unprecedented memory compression.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a dominant novel-view synthesis technique, but its high memory consumption severely limits its applicability on edge devices. A growing number of 3DGS compression methods have been proposed to make 3DGS more efficient, yet most only focus on storage compression and fail to address the critical bottleneck of rendering memory. To address this problem, we introduce MEGS$^{2}$, a novel memory-efficient framework that tackles this challenge by jointly optimizing two key factors: the total primitive number and the parameters per primitive, achieving unprecedented memory compression. Specifically, we replace the memory-intensive spherical harmonics with lightweight, arbitrarily oriented spherical Gaussian lobes as our color representations. More importantly, we propose a unified soft pruning framework that models primitive-number and lobe-number pruning as a single constrained optimization problem. Experiments show that MEGS$^{2}$ achieves a 50% static VRAM reduction and a 40% rendering VRAM reduction compared to existing methods, while maintaining comparable rendering quality. Project page: https://megs-2.github.io/

MEGS$^{2}$: Memory-Efficient Gaussian Splatting via Spherical Gaussians and Unified Pruning

TL;DR

MEGS is introduced, a novel memory-efficient framework that tackles the challenge of memory compression in 3D Gaussian Splatting by jointly optimizing two key factors: the total primitive number and the parameters per primitive, achieving unprecedented memory compression.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a dominant novel-view synthesis technique, but its high memory consumption severely limits its applicability on edge devices. A growing number of 3DGS compression methods have been proposed to make 3DGS more efficient, yet most only focus on storage compression and fail to address the critical bottleneck of rendering memory. To address this problem, we introduce MEGS, a novel memory-efficient framework that tackles this challenge by jointly optimizing two key factors: the total primitive number and the parameters per primitive, achieving unprecedented memory compression. Specifically, we replace the memory-intensive spherical harmonics with lightweight, arbitrarily oriented spherical Gaussian lobes as our color representations. More importantly, we propose a unified soft pruning framework that models primitive-number and lobe-number pruning as a single constrained optimization problem. Experiments show that MEGS achieves a 50% static VRAM reduction and a 40% rendering VRAM reduction compared to existing methods, while maintaining comparable rendering quality. Project page: https://megs-2.github.io/

Paper Structure

This paper contains 29 sections, 31 equations, 14 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Evolution of Splatting-Based Scene Representations
  4. Memory Analysis in 3DGS Rendering
  5. Pruning Techniques for 3DGS
  6. Other Compression Schemes for 3DGS
  7. Method
  8. Preliminaries for 3D Gaussian Splatting
  9. Spherical Gaussians
  10. Unified soft-pruning framework
  11. Problem Formulation
  12. Memory-constrained Optimization
  13. Post-Processing Procedure
  14. Experiments
  15. Experimental Settings
  16. ...and 14 more sections

Figures (14)

  • Figure 1: We present MEGS2, a memory-efficient framework designed to solve the rendering memory bottleneck of 3D Gaussian Splatting and enable high-quality, real-time rendering on edge devices. As demonstrated in our WebGL-based viewer, 3D Gaussian Splatting(3DGS) with Spherical Harmonics (SH) exhibits low frame rates on desktop GPU and fails to run on some mobile platforms. In contrast, MEGS2 achieves interactive frame rates across all tested devices, significantly expanding the applicability of 3DGS. The detailed result is in Appendix \ref{['sec: appendix_quantitive_res']} and \ref{['sec:appendix_qualitative_res']}.
  • Figure 2: Overview of our proposed MEGS2. (A) In Section \ref{['sec:sg']}, we first replace the Spherical Harmonics with arbitrarily-oriented and prunable Spherical Gaussians. (B) In Section \ref{['sec:pf']}, we formulate the compression as a memory-constrained optimization problem, which is solved using an ADMM-inspired approach (Section \ref{['sec:algo']}). (C) In Section \ref{['sec:postprocessing']}, near-invalid primitives and low-sharpness lobes are removed, and a color compensation term (Eq.\ref{['equation:compensation']}) is introduced to recover the energy of the removed lobes.
  • Figure 3: Qualitative results on the Bicycle, Bonsai, Kitchen, Playroom and Truck scenes comparing to previous methods and the corresponding ground truth images from test views. The rendering VRAM consumption for the corresponding method is annotated at the bottom of each image.
  • Figure 4: Comparison of different 3DGS color representations on Mip-NeRF360 dataset.
  • Figure 5: Distribution of Gaussian lobes across scenes.
  • ...and 9 more figures