CompGS++: Compressed Gaussian Splatting for Static and Dynamic Scene Representation

TL;DR

CompGS++ introduces a compression framework for Gaussian Splatting that jointly targets static and dynamic scene representation. It combines a spatial primitive prediction module that exploits inter-primitive correlations with a rate-constrained optimization that balances distortion and bitrate, using an entropy-model with spatial priors and hyperpriors. For dynamic scenes, it extends to temporal primitive prediction and a temporal adaptive control that disentangles dynamic versus static primitives and creates new primitives as needed, enabling compact 4D scene representations. Across static and dynamic benchmarks, CompGS++ achieves substantial compression gains with rendering quality comparable to or better than state-of-the-art methods, enabling efficient transmission and progressive streaming of photorealistic 3D content over the Internet.

Abstract

Gaussian splatting demonstrates proficiency for 3D scene modeling but suffers from substantial data volume due to inherent primitive redundancy. To enable future photorealistic 3D immersive visual communication applications, significant compression is essential for transmission over the existing Internet infrastructure. Hence, we propose Compressed Gaussian Splatting (CompGS++), a novel framework that leverages compact Gaussian primitives to achieve accurate 3D modeling with substantial size reduction for both static and dynamic scenes. Our design is based on the principle of eliminating redundancy both between and within primitives. Specifically, we develop a comprehensive prediction paradigm to address inter-primitive redundancy through spatial and temporal primitive prediction modules. The spatial primitive prediction module establishes predictive relationships for scene primitives and enables most primitives to be encoded as compact residuals, substantially reducing the spatial redundancy. We further devise a temporal primitive prediction module to handle dynamic scenes, which exploits primitive correlations across timestamps to effectively reduce temporal redundancy. Moreover, we devise a rate-constrained optimization module that jointly minimizes reconstruction error and rate consumption. This module effectively eliminates parameter redundancy within primitives and enhances the overall compactness of scene representations. Comprehensive evaluations across multiple benchmark datasets demonstrate that CompGS++ significantly outperforms existing methods, achieving superior compression performance while preserving accurate scene modeling. Our implementation will be made publicly available on GitHub to facilitate further research.

Figures (13)

• Figure 1: Illustration of spatial correlations of 3D Gaussians. Left: Scene visualization having enlarged view of objects with inherent continuity. Right: Demonstration of primitive correlations arising from the scene continuity.
• Figure 2: Statistical analysis of spatial correlations between 3D Gaussians, wherein spatial correlations are measured by the average cosine distances between a 3D Gaussian and its 20 nearest neighbors.
• Figure 3: Visualization of temporal correlations between 3D Gaussians. Left: Previous P-scene with enlarged view of 3D Gaussians in the dynamic area. Right: Subsequent P-scene with visualized 3D Gaussians in the same dynamic area, demonstrating notable temporal correlations of 3D Gaussians.
• Figure 4: Illustration of temporal shifts in ambient illumination. Left: Static regions of the current P-scene. Right: Difference between the current and subsequent P-scenes, revealing subtle temporal ambient illumination variations in static regions.
• Figure 5: Overview of the proposed compact representation framework for static scenes.