SuperGaussians: Enhancing Gaussian Splatting Using Primitives with Spatially Varying Colors

TL;DR

<1> The paper tackles the inefficiency of Gaussian splatting representations that rely on a single view-dependent color and opacity per primitive, which struggles to capture complex textures and geometry. <2> It introduces SuperGaussians, enabling spatially varying colors and opacity on Gaussian surfels via three designs: bilinear interpolation, movable kernels, and tiny MLPs, to enhance expressiveness and reconstruction quality. <3> Across synthetic and real datasets, all three designs outperform the baseline 2DGS in novel-view synthesis, with movable kernels delivering the best performance, and with competitive or superior geometry when Gaussian budgets are limited. <4> The work also includes ablations and discusses limitations, such as slower training times, while pointing to future work on additional spatial variation functions and extension to Gaussian ellipsoids for richer scene representations.

Abstract

Gaussian Splattings demonstrate impressive results in multi-view reconstruction based on Gaussian explicit representations. However, the current Gaussian primitives only have a single view-dependent color and an opacity to represent the appearance and geometry of the scene, resulting in a non-compact representation. In this paper, we introduce a new method called SuperGaussians that utilizes spatially varying colors and opacity in a single Gaussian primitive to improve its representation ability. We have implemented bilinear interpolation, movable kernels, and even tiny neural networks as spatially varying functions. Quantitative and qualitative experimental results demonstrate that all three functions outperform the baseline, with the best movable kernels achieving superior novel view synthesis performance on multiple datasets, highlighting the strong potential of spatially varying functions.

Figures (16)

• Figure 1: SuperGaussians gives each Gaussian the ability to vary spatially. Compared with 2DGS 2dgs and 3DGS 3dgs, SuperGaussians is more expressive and can better reconstruct details (as seen in the white area). Additionally, SuperGaussians can express more color variations using only one Gaussian, instead of being limited to just one color (bottom right).
• Figure 2: 3DGS 3dgs uses Gaussian ellipsoids to express scenes, and a learnable color is defined on each ellipsoid. 2DGS 2dgs uses Gaussian surfels to express scenes, and a learnable color is defined on each Gaussian surfel. Our SuperGaussians uses spatially varying Gaussian surfels to express scenes, and the color and opacity changes with the spatial position on each surfel.
• Figure 3: Visualization of the bilinear interpolation (a,c) and movable kernel functions (b,d).
• Figure 4: The parameter amounts of the three proposed spatial variation functions and the original 2DGS 2dgs on a single Gaussian. The parameter amounts of the three proposed functions are 1.28 times, 1.40 times, and 1.88 times of the original 2DGS 2dgs.
• Figure 5: Visual comparison between three different spatially varying functions and 2DGS 2dgs.