3DGEER: 3D Gaussian Rendering Made Exact and Efficient for Generic Cameras

TL;DR

3DGEER tackles the fundamental mismatch between 3D Gaussian representations and real-world camera projections by deriving a geometrically exact, differentiable rendering framework for arbitrary camera models. It achieves real-time performance via a novel Ray-Particle association (PBF) and a camera-agnostic BEAP sampling scheme, while maintaining exact projective integrals along rays. The approach yields strong improvements across wide-FoV and fisheye datasets, outperforming prior projective-approximate methods and enabling better generalization to unseen viewpoints. This work significantly advances real-time radiance-field rendering for generic cameras with both high fidelity and broad applicability in robotics, AR/VR, and autonomous systems.

Abstract

3D Gaussian Splatting (3DGS) achieves an appealing balance between rendering quality and efficiency, but relies on approximating 3D Gaussians as 2D projections--an assumption that degrades accuracy, especially under generic large field-of-view (FoV) cameras. Despite recent extensions, no prior work has simultaneously achieved both projective exactness and real-time efficiency for general cameras. We introduce 3DGEER, a geometrically exact and efficient Gaussian rendering framework. From first principles, we derive a closed-form expression for integrating Gaussian density along a ray, enabling precise forward rendering and differentiable optimization under arbitrary camera models. To retain efficiency, we propose the Particle Bounding Frustum (PBF), which provides tight ray-Gaussian association without BVH traversal, and the Bipolar Equiangular Projection (BEAP), which unifies FoV representations, accelerates association, and improves reconstruction quality. Experiments on both pinhole and fisheye datasets show that 3DGEER outperforms prior methods across all metrics, runs 5x faster than existing projective exact ray-based baselines, and generalizes to wider FoVs unseen during training--establishing a new state of the art in real-time radiance field rendering.

Figures (18)

• Figure 1: Linear Approximation Error in Ray-Particle Association. (Left) Grid-line artifacts caused by inaccurate UT association. (Right) Diagram illustrating our association and comparison with others. Our method avoids the intermediate conic approximation by directly computing the exact bounding structure from the true 3D covariance.
• Figure 2: Gradient Flow and Canonical Transformation. (Left) Illustration of forward and backward gradient propagation from the Gaussian parameters $\mathbf{s}$, $\mathbf{q}$, $\boldsymbol{\mu}$, and the opacity coefficient $\sigma$ to the transmittance. (Right) Integral of 3D Gaussian density along a ray, where the green box highlights the product of density and ray segment length (see Eq. \ref{['eq:integral:same']}).
• Figure 3: PBF-CSF Association. (Left) PBF defined by four planes tangent to a 3D Gaussian ellipsoid. (Right) The intersection between the PBF and CSF, unfolded onto the BEAP imaging plane.
• Figure 4: Comparison of Ray Distributions under Varying Projections. We project pixels from three imaging spaces onto the unit sphere to compare ray distributions. Among them, our BEAP projection achieves the most uniform coverage.
• Figure 5: PSNR$\uparrow$ Trends When Scaling-Up Gaussian Counts. Brute-force scaling fails to close the gap in projective exactness (see full metrics in Fig. \ref{['fig:scaling-full']} and Tab. \ref{['tab:scannet-scaling-details']}).

Theorems & Definitions (2)

• Theorem 1
• proof