Bayesian NeRF: Quantifying Uncertainty with Volume Density for Neural Implicit Fields

Abstract

We present a Bayesian Neural Radiance Field (NeRF), which explicitly quantifies uncertainty in the volume density by modeling uncertainty in the occupancy, without the need for additional networks, making it particularly suited for challenging observations and uncontrolled image environments. NeRF diverges from traditional geometric methods by providing an enriched scene representation, rendering color and density in 3D space from various viewpoints. However, NeRF encounters limitations in addressing uncertainties solely through geometric structure information, leading to inaccuracies when interpreting scenes with insufficient real-world observations. While previous efforts have relied on auxiliary networks, we propose a series of formulation extensions to NeRF that manage uncertainties in density, both color and density, and occupancy, all without the need for additional networks. In experiments, we show that our method significantly enhances performance on RGB and depth images in the comprehensive dataset. Given that uncertainty modeling aligns well with the inherently uncertain environments of Simultaneous Localization and Mapping (SLAM), we applied our approach to SLAM systems and observed notable improvements in mapping and tracking performance. These results confirm the effectiveness of our Bayesian NeRF approach in quantifying uncertainty based on geometric structure, making it a robust solution for challenging real-world scenarios.

Figures (5)

• Figure 1: Quantitative results. Ground Truth (Left), baseline model prediction (Middle), our method's prediction (Right). All models were trained on 4 images from the NeRF dataset and predict unobserved viewpoints.
• Figure 2: Dataset Settings. We conduct experiments on NeRF synthetic, real-world, and ModelNet datasets, using green cameras for training and blue cameras for testing. (a) The NeRF synthetic dataset evaluates unobserved predictions with limited training views, while the real-world dataset assesses observed predictions with forward-facing camera trajectories. (b) The ModelNet dataset evaluates both unobserved and observed predictions using 36 images spaced at 10-degree intervals.
• Figure 3: Qualitative Comparisons on NeRF synthetic and LLFF datasets mildenhall2021nerf. The Chair scene is from the NeRF synthetic dataset trained with the syn4 setting, and the T-rex scene is from the LLFF dataset trained with the real8 setting. These settings are described in \ref{['fig: dataset setting']}. While incorporating uncertainty improves performance, the (b) Color method in synthetic datasets struggles with density estimation, causing objects to appear faded.
• Figure 4: Qualitative Comparisons on ModelNet wu20153d. The sofa and dresser scene employs an unobserved view setting and is trained using 8 images. In this scene, the (d) Den_cf and (f) Occupancy methods effectively addresses the blurring issue.
• Figure 5: Mapping and Tracking Visualization Results on the Replica Office2 scene. The figure shows intermediate rendering results during the mapping process, which are not used for training. Our method demonstrates improvements over the original method in both mapping and tracking, even with limited data.