P-GSVC: Layered Progressive 2D Gaussian Splatting for Scalable Image and Video

TL;DR

P-GSVC is presented, the first layered progressive 2D Gaussian splatting framework that provides a unified solution for scalable Gaussian representation in both images and videos and supports scalability in terms of both quality and resolution.

Abstract

Gaussian splatting has emerged as a competitive explicit representation for image and video reconstruction. In this work, we present P-GSVC, the first layered progressive 2D Gaussian splatting framework that provides a unified solution for scalable Gaussian representation in both images and videos. P-GSVC organizes 2D Gaussian splats into a base layer and successive enhancement layers, enabling coarse-to-fine reconstructions. To effectively optimize this layered representation, we propose a joint training strategy that simultaneously updates Gaussians across layers, aligning their optimization trajectories to ensure inter-layer compatibility and a stable progressive reconstruction. P-GSVC supports scalability in terms of both quality and resolution. Our experiments show that the joint training strategy can gain up to 1.9 dB improvement in PSNR for video and 2.6 dB improvement in PSNR for image when compared to methods that perform sequential layer-wise training. Project page: https://longanwang-cs.github.io/PGSVC-webpage/

Figures (8)

• Figure 1: Progressive reconstruction of (a) Pruned Gaussian splats and (b) Layered Gaussian splats (our P-GSVC). Both columns use the same number of splats, and the quality improves from left to right as more splats are added. Noticeable holes appear in (a), while (b) preserves scene integrity.
• Figure 2: Comparison of gradient and loss over iterations on single-layer and multi-layer optimization strategies: (a) Single-layer optimization in GSVC; (b) Multi-layer trained sequentially in GSVC (the targeted layer is switched at the 50k iteration and the 100k iteration, while the previously trained layers remain frozen). One can observe multi-layer training has issues of unstable convergence and suboptimal local minima.
• Figure 3: The architecture of P-GSVC, shown with three layers as an example. The base layer $L_0$ provides the coarse representation, while the enhancement layers $\Delta L_1$ and $\Delta L_2$ progressively add additional 2D Gaussian splats. By incrementally incorporating these enhancement layers, the video quality is refined progressively, yielding higher-quality/resolution reconstructions.
• Figure 4: Progressive procedure of layered Gaussians. $\Delta\boldsymbol{\mathcal{G}}^{1}$ and $\Delta\boldsymbol{\mathcal{G}}^{2}$ capture higher-frequency details and can be iteratively added to $\boldsymbol{\mathcal{G}}^{0}$, yielding progressively refined $\boldsymbol{\mathcal{G}}^{2}$.
• Figure 5: Comparison of gradient and loss over iterations on randomly and cyclically joint training strategies: jointly optimizes layers (a) randomly and (b) in a cyclic order.