EGGS: Exchangeable 2D/3D Gaussian Splatting for Geometry-Appearance Balanced Novel View Synthesis

TL;DR

EGGS introduces an exchangeable hybrid Gaussian Splatting framework that unifies 2D and 3D Gaussian representations to balance geometric accuracy and appearance fidelity in novel view synthesis. It combines Hybrid Gaussian Rasterization, Adaptive Type Exchange guided by effective rank, and Frequency-Decoupled Optimization using Discrete Wavelet Transform to decouple geometry and appearance supervision. Across diverse benchmarks, EGGS achieves superior rendering quality and geometry while maintaining efficiency, and demonstrates robustness in few-shot and out-of-distribution conditions. The approach provides a flexible, CUDA-accelerated framework that can integrate with existing 3DGS and 2DGS pipelines to improve NVS and 3D reconstruction.

Abstract

Novel view synthesis (NVS) is crucial in computer vision and graphics, with wide applications in AR, VR, and autonomous driving. While 3D Gaussian Splatting (3DGS) enables real-time rendering with high appearance fidelity, it suffers from multi-view inconsistencies, limiting geometric accuracy. In contrast, 2D Gaussian Splatting (2DGS) enforces multi-view consistency but compromises texture details. To address these limitations, we propose Exchangeable Gaussian Splatting (EGGS), a hybrid representation that integrates 2D and 3D Gaussians to balance appearance and geometry. To achieve this, we introduce Hybrid Gaussian Rasterization for unified rendering, Adaptive Type Exchange for dynamic adaptation between 2D and 3D Gaussians, and Frequency-Decoupled Optimization that effectively exploits the strengths of each type of Gaussian representation. Our CUDA-accelerated implementation ensures efficient training and inference. Extensive experiments demonstrate that EGGS outperforms existing methods in rendering quality, geometric accuracy, and efficiency, providing a practical solution for high-quality NVS.

Figures (15)

• Figure 1: Comparison of 3DGS, 2DGS, and our EGGS. While 3DGS achieves high-fidelity appearance, it often produces inaccurate geometry, with imprecise surfaces and blurred edges. 2DGS improves geometric consistency across views but suffers from reduced appearance quality due to over-smoothed surfaces and loss of detail. In contrast, EGGS employs an exchangeable hybrid Gaussian representation that achieves both accurate geometry and high-quality appearance.
• Figure 2: Left: Comparison of 3DGS and 2DGS in appearance and geometry metrics. Right: Comparison between EGGS and related works. Prior works either use only single representation or do not explore complementary advantages of 3D and 2D Gaussians. $^\star$ Gaussian Surfel dai2024surfles directly sets the $z$-scale of 3D Gaussian to zero and uses the rasterizer from 3DGS. * HybridGS lin2024hybridgs uses image-frame single-view 2D Gaussians zhang2024imagezhang2024gaussianimage instead of 2D Gaussians in the 3D space 2dgs.
• Figure 3: Overview of the EGGS framework. We initialize 2D and 3D Gaussians from sparse points obtained via structure-from-motion (SfM) schonberger2016colmapschonberger2016colmap2. Their parameters are then jointly optimized using our CUDA-accelerated differentiable hybrid rasterization. To enhance the flexibility of the hybrid representation, Adaptive Type Exchange is introduced to allow each Gaussian to switch between 2D and 3D types during training. Finally, we apply Discrete Wavelet Transform (DWT) heil1989dwt and introduce Frequency-Decoupled Optimization to balance geometric accuracy and appearance fidelity.
• Figure 3: Comparison on efficiency. # Gaussians is the average number of Gaussians.
• Figure 4: Illustration of Hybrid Gaussian Rasterization. The contribution of 3D Gaussians and 2D Gaussians is computed via affine projection and ray-splat-intersection, respectively.