Compact 3D Scene Representation via Self-Organizing Gaussian Grids

TL;DR

3D Gaussian Splatting offers fast, high-quality novel-view rendering but at a prohibitive storage cost. The authors convert unconstrained Gaussian parameters into a structured 2D grid using a parallel sorting method (PLAS) and enforce local smoothness during training, followed by quantization and image-based compression. This workflow yields a 17x–42x reduction in model size across real and synthetic scenes while preserving rendering quality and keeping training time near vanilla 3DGS. The approach enables efficient 3D scene distribution and real-time rendering on devices with limited memory and bandwidth, with potential extensions to dynamic scenes.

Abstract

3D Gaussian Splatting has recently emerged as a highly promising technique for modeling of static 3D scenes. In contrast to Neural Radiance Fields, it utilizes efficient rasterization allowing for very fast rendering at high-quality. However, the storage size is significantly higher, which hinders practical deployment, e.g. on resource constrained devices. In this paper, we introduce a compact scene representation organizing the parameters of 3D Gaussian Splatting (3DGS) into a 2D grid with local homogeneity, ensuring a drastic reduction in storage requirements without compromising visual quality during rendering. Central to our idea is the explicit exploitation of perceptual redundancies present in natural scenes. In essence, the inherent nature of a scene allows for numerous permutations of Gaussian parameters to equivalently represent it. To this end, we propose a novel highly parallel algorithm that regularly arranges the high-dimensional Gaussian parameters into a 2D grid while preserving their neighborhood structure. During training, we further enforce local smoothness between the sorted parameters in the grid. The uncompressed Gaussians use the same structure as 3DGS, ensuring a seamless integration with established renderers. Our method achieves a reduction factor of 17x to 42x in size for complex scenes with no increase in training time, marking a substantial leap forward in the domain of 3D scene distribution and consumption. Additional information can be found on our project page: https://fraunhoferhhi.github.io/Self-Organizing-Gaussians/

Figures (10)

• Figure 1: Our 3DGS training method allows for high compression of 3DGS attributes while maintaining high rendering quality. By sorting the Gaussian features periodically into a 2D grid and applying a smoothness loss during scene generation, we significantly reduce the storage footprint using state-of-the-art image compression methods.
• Figure 1: Behavior of the sorting algorithm under different break thresholds. Decreasing the relative L2 threshold trades off additional sorting quality for longer runtime. These values were measured with an NVidia RTX 4090 on a random 512x512x3 grid. In all our training experiments, we fixed this parameter to $10^{-4}$.
• Figure 2: An overview of our novel 3DGS training method. During training, we arrange all high dimensional attributes into multiple 2D grids. Those grids are sorted and a smoothness regularization is applied. This creates redundancy which help to compress the 2D grids into small files using off-the-shelf compression methods.
• Figure 3: Visualization of the conversion of 2D attribute grids into Gaussian splat objects. Note that the sorted grids have to align. Therefore, they cannot be sorted individually. Here, color represents the DC components of the spherical harmonics.
• Figure 3: A comparison between the rendering quality when only applying our compression method to the vanilla 3DGS model without smoothing.