Extreme Views: 3DGS Filter for Novel View Synthesis from Out-of-Distribution Camera Poses

TL;DR

This work tackles the problem of visual artifacts in 3D Gaussian Splatting (3DGS) when rendering from camera poses outside the training distribution. It introduces a render-time gradient-sensitivity analysis with a rotation-aligned coordinate frame, executed as a two-pass filter that prunes unstable Gaussians whose aggregate rejection exceeds a threshold $\tau_{ratio}$, guided by per-intersection sensitivity $\tau_{grad}$. The method leverages ray marching in 3D space and does not require retraining, yielding significant perceptual improvements over baselines like BayesRays while preserving real-time performance. Practically, the approach enhances robustness and realism for interactive novel-view synthesis in out-of-distribution trajectories.

Abstract

When viewing a 3D Gaussian Splatting (3DGS) model from camera positions significantly outside the training data distribution, substantial visual noise commonly occurs. These artifacts result from the lack of training data in these extrapolated regions, leading to uncertain density, color, and geometry predictions from the model. To address this issue, we propose a novel real-time render-aware filtering method. Our approach leverages sensitivity scores derived from intermediate gradients, explicitly targeting instabilities caused by anisotropic orientations rather than isotropic variance. This filtering method directly addresses the core issue of generative uncertainty, allowing 3D reconstruction systems to maintain high visual fidelity even when users freely navigate outside the original training viewpoints. Experimental evaluation demonstrates that our method substantially improves visual quality, realism, and consistency compared to existing Neural Radiance Field (NeRF)-based approaches such as BayesRays. Critically, our filter seamlessly integrates into existing 3DGS rendering pipelines in real-time, unlike methods that require extensive post-hoc retraining or fine-tuning. Code and results at https://damian-bowness.github.io/EV3DGS

Figures (6)

• Figure 1: Novel view synthesis from an OOD camera pose(red) compared to the training cameras poses (blue).
• Figure 2: Pipeline of our two-pass filter
• Figure 3: Deep Blending example frames.
• Figure 4: NeRF-On-the-Go samples highlight our filter’s suppression of ambiguous regions from in-distribution camera poses.
• Figure 5: (a) Single-pass render of Playroom scene. $\tau_{grad.}=10^{-5}$ (b) and (c) Loss of detail when including scale (b) vs. no scale (c)