ProtoGS: Efficient and High-Quality Rendering with 3D Gaussian Prototypes

TL;DR

ProtoGS addresses the memory bottleneck of 3D Gaussian Splatting by learning a compact set of Gaussian prototypes that represent nearby primitives. It introduces SfM anchoring to tile primitives and a rendering-guided K-means objective to derive prototypes within each tile, with joint optimization that preserves geometry and texture while replacing primitives with prototypes. Unlike methods relying on implicit MLPs, ProtoGS maintains fast rendering and high fidelity across diverse datasets, achieving substantial reductions in primitive counts and storage while boosting rendering speed. The results demonstrate strong compression-quality trade-offs and practical potential for real-time 3D scene rendering in AR/VR contexts.

Abstract

3D Gaussian Splatting (3DGS) has made significant strides in novel view synthesis but is limited by the substantial number of Gaussian primitives required, posing challenges for deployment on lightweight devices. Recent methods address this issue by compressing the storage size of densified Gaussians, yet fail to preserve rendering quality and efficiency. To overcome these limitations, we propose ProtoGS to learn Gaussian prototypes to represent Gaussian primitives, significantly reducing the total Gaussian amount without sacrificing visual quality. Our method directly uses Gaussian prototypes to enable efficient rendering and leverage the resulting reconstruction loss to guide prototype learning. To further optimize memory efficiency during training, we incorporate structure-from-motion (SfM) points as anchor points to group Gaussian primitives. Gaussian prototypes are derived within each group by clustering of K-means, and both the anchor points and the prototypes are optimized jointly. Our experiments on real-world and synthetic datasets prove that we outperform existing methods, achieving a substantial reduction in the number of Gaussians, and enabling high rendering speed while maintaining or even enhancing rendering fidelity.

Figures (8)

• Figure 1: We propose a novel method that significantly reduces the memory footprint of 3D Gaussian Splatting, meanwhile enjoys excellent visual quality and high rendering speed. Please zoom in for better clarity, and more results are in supplementary material.
• Figure 2: Overview of our method. Starting with Gaussian primitives (denoted by green ellipsoids), we first assign primitives to multiple tiles (distinguished by different colors) centered by SfM points as shown in (a), then we derive 3D Gaussian prototypes from primitives within each tile in (b) controlled by a compression ratio. After that, we utilize rasterization to render images based on prototypes, and propagate gradients back to update Gaussian primitives iteratively as shown in (d). After multiple processes (d), we replace current primitives with prototypes to reduce the total number of primitives.
• Figure 3: From the second to the last row, we show qualitative comparison of ProtoGS (Ours), 3DGS, ScaffoldGS, Eagles, CompactGS, and LightGS . We show the corresponding ground truth images on the first row, from the left to right: Drjohnson, Playroom from the deep blending dataset; and Truck and Train from Tanks&Temples. Visual differences are highlighted by green frames and red arrow and corresponding regions are zoomed in and put in the bottom left corner of each image.
• Figure 4: Visualization of ellipsoids. We visualize ellipsoids learned by our method and 3DGS on Drjohnson (first row) and Playroom scenes (second and third row). We use green frames to show that our method successfully circumvents redundant finely chopped primitives like 3DGS, and the zoomed-in regions are shown in the bottom right corners of each image.
• Figure 5: Comparison of our method with other competitors in average storage size and rendering speed over all datasets.