Optimize the Unseen -- Fast NeRF Cleanup with Free Space Prior

TL;DR

NeRFs suffer floaters in unseen regions due to photometric optimization. The paper introduces a fast post-hoc cleanup that enforces a Free Space Prior by sampling across the full 3D space and optimizing the density $\sigma$ toward zero in unseen regions, integrated via $\mathcal{L} = \mathcal{L}_{rec} + \lambda \mathcal{L}_{FSP}$ with $\lambda=0.1$. This MAP-based, global-prior approach is architecture-agnostic and yields cleanup results with high coverage and PSNR while being $2.5\times$ faster at inference and requiring no extra memory, training in under 30 seconds. Empirically, it generalizes across NeRF variants and datasets, offering a practical, scalable solution for artifact removal that preserves scene integrity and improves novel-view synthesis quality.

Abstract

Neural Radiance Fields (NeRF) have advanced photorealistic novel view synthesis, but their reliance on photometric reconstruction introduces artifacts, commonly known as "floaters". These artifacts degrade novel view quality, especially in areas unseen by the training cameras. We present a fast, post-hoc NeRF cleanup method that eliminates such artifacts by enforcing our Free Space Prior, effectively minimizing floaters without disrupting the NeRF's representation of observed regions. Unlike existing approaches that rely on either Maximum Likelihood (ML) estimation to fit the data or a complex, local data-driven prior, our method adopts a Maximum-a-Posteriori (MAP) approach, selecting the optimal model parameters under a simple global prior assumption that unseen regions should remain empty. This enables our method to clean artifacts in both seen and unseen areas, enhancing novel view quality even in challenging scene regions. Our method is comparable with existing NeRF cleanup models while being 2.5x faster in inference time, requires no additional memory beyond the original NeRF, and achieves cleanup training in less than 30 seconds. Our code will be made publically available.

Figures (13)

• Figure 1: Density Cleanup Across NeRF Variants and Datasets: The top row shows novel views taken near the training cameras, while the middle row presents novel views from farther away. The bottom row demonstrates the same distant views after applying our cleanup method, effectively removing artifacts while preserving scene integrity. Our approach generalizes well across a wide range of NeRF methods and datasets - see supplementary material for video demonstrations.
• Figure 2: Overview of Our Cleanup Method: Our approach fine-tunes a pre-trained NeRF by optimizing density across both seen and unseen regions. We sample points in two ways: (1) along the original training rays (green points), maintaining consistency with the observed scene structure, and (2) randomly across the entire 3D space (blue points), enforcing our Free Space Prior to remove artifacts in unseen areas. Importantly, we also apply the Free Space Prior behind scene surfaces, ensuring that empty regions remain free of unwanted density accumulations. This ensures effective cleanup of floaters while preserving scene integrity.
• Figure 3: Qualitative Cleanup Results: Comparison of cleanup methods on the Plant and Roses scenes from the Nerfbusters dataset, with inference time per frame shown. Each image is a novel view rendered post-cleanup, highlighting the balance between artifact removal and scene coverage. Methods like Free Space Prior and BayesRays (0.9) achieve high coverage with minimal artifacts, while Nerfbusters and BayesRays (0.3) trade coverage for stronger cleanup. Our approach achieves similar results to BayesRays with 40% rendering time.
• Figure 4: Density Along a Ray: (Left) Novel view rendered with the Nerfacto Baseline, with a red dot marking the sampled ray and the density along the marked ray. (Right) The same novel view after our cleanup. Before cleanup, the density along the ray is noisy, with floaters causing unintended peaks. After cleanup, only the surface is captured, and densities in unseen regions are enforced to zero. Densities are sigmoid softened and shifted to [0,1].
• Figure 5: Quantitative Cleanup Results: PSNR vs. Coverage comparison of cleanup methods across different thresholds / $\lambda$. Higher values indicate better performance, with the optimal region in the upper-right corner. Results are averaged across all Nerfbusters scenes for a comprehensive evaluation.