ProGS: Towards Progressive Coding for 3D Gaussian Splatting

TL;DR

The proposed ProGS is a streaming-friendly codec that facilitates progressive coding for 3D Gaussian splatting, and significantly improves both compression efficiency and visual fidelity, demonstrating that ProGS can provide a robust solution for real-time applications with varying network conditions.

Abstract

With the emergence of 3D Gaussian Splatting (3DGS), numerous pioneering efforts have been made to address the effective compression issue of massive 3DGS data. 3DGS offers an efficient and scalable representation of 3D scenes by utilizing learnable 3D Gaussians, but the large size of the generated data has posed significant challenges for storage and transmission. Existing methods, however, have been limited by their inability to support progressive coding, a crucial feature in streaming applications with varying bandwidth. To tackle this limitation, this paper introduce a novel approach that organizes 3DGS data into an octree structure, enabling efficient progressive coding. The proposed ProGS is a streaming-friendly codec that facilitates progressive coding for 3D Gaussian splatting, and significantly improves both compression efficiency and visual fidelity. The proposed method incorporates mutual information enhancement mechanisms to mitigate structural redundancy, leveraging the relevance between nodes in the octree hierarchy. By adapting the octree structure and dynamically adjusting the anchor nodes, ProGS ensures scalable data compression without compromising the rendering quality. ProGS achieves a remarkable 45X reduction in file storage compared to the original 3DGS format, while simultaneously improving visual performance by over 10%. This demonstrates that ProGS can provide a robust solution for real-time applications with varying network conditions.

Figures (9)

• Figure 1: ProGS structures a 3D scene into a group of octrees with multiple layers. Each level-of-detail (LoD) $l$ of the whole scene can be viewed as the first $l$ layers of the octrees. With context-based progressive coding, ProGS can be delivered and decoded in scalable bitrate, enabling adaptability for bandwidth variations and error conditions.
• Figure 2: Overview of the proposed ProGS. ProGS constructs an initial octree group using the input SfM point cloud. The $l$-th LoD of ProGS contains all anchors with levels beneath $l$. During training, ProGS adaptively adjusts the anchors according to the optimization gradients. For clear illustration, here uses quadtree of 2D planes to demonstrate the idea of tree-structured anchors.
• Figure 3: Illustration of the anchor adjustment strategy. This figure takes 2D quadtree planes as examples. (i) Candidate $c_n$ placed at the same level of Gaussian $G_m$. (ii) Candidates $c_j$ and $c_q$ placed at the upper level of Gaussians $G_i$ and $G_q$. With no parent anchor for $c_j$, $c_{p_j}$ is queued. (iii) The pruning operation.
• Figure 4: RD curves. Notably, the solid lines are the RD curves of different bitrate versions obtained by changing different $\lambda_\text{e}$ at the same LoD, while the dotted lines are the RD curves of the same bitrate version at different LoDs.
• Figure 5: The training process of ProGS.