Evidential Neural Radiance Fields

TL;DR

This work introduces Evidential Neural Radiance Fields, a probabilistic approach that seamlessly integrates with the NeRF rendering process and enables direct quantification of both aleatoric and epistemic uncertainty from a single forward pass, and compares multiple uncertainty quantification methods on three standardized benchmarks.

Abstract

Understanding sources of uncertainty is fundamental to trustworthy three-dimensional scene modeling. While recent advances in neural radiance fields (NeRFs) achieve impressive accuracy in scene reconstruction and novel view synthesis, the lack of uncertainty estimation significantly limits their deployment in safety-critical settings. Existing uncertainty quantification methods for NeRFs fail to capture both aleatoric and epistemic uncertainty. Among those that do quantify one or the other, many of them either compromise rendering quality or incur significant computational overhead to obtain uncertainty estimates. To address these issues, we introduce Evidential Neural Radiance Fields, a probabilistic approach that seamlessly integrates with the NeRF rendering process and enables direct quantification of both aleatoric and epistemic uncertainty from a single forward pass. We compare multiple uncertainty quantification methods on three standardized benchmarks, where our approach demonstrates state-of-the-art scene reconstruction fidelity and uncertainty estimation quality.

Figures (14)

• Figure 1: Evolution of NeRF pipeline across three levels of probabilistic formulations. $N$ voxels sampled along the camera ray give $N$ pairs of spatial location and view direction, which are passed to the NeRF model for prediction. Level 1: Vanilla NeRF predicts only voxel density and color, resulting in a point estimate of pixel color without any uncertainty estimate. Level 2: Normal NeRF assumes the voxel and pixel colors follow normal distributions, quantifying aleatoric uncertainty of rendered color. Level 3: Evidential NeRF assumes the pixel color has random mean and variance following an evidential distribution, quantifying both aleatoric and epistemic uncertainties.
• Figure 2: Qualitative comparison against previous methods on two example scenes, with image reconstructions, error maps, and uncertainty maps. Histogram equalization is conducted on the error maps to highlight the error regions. Our method's uncertainty is total uncertainty. In general, our total uncertainty maps demonstrate superior alignment with predictive errors, confirming that the joint modeling of aleatoric and epistemic uncertainties yields the most faithful uncertainty estimates. More qualitative figures are provided in \ref{['sec:qualitative']}.
• Figure 3: Test AU and EU vs. training sample size. AU remains stable while EU significantly declines with more data observed.
• Figure 4: Aleatoric and epistemic uncertainties of a scene in the wild. AU arises from intrinsic data variability captured in the training set, including radiance variation (e.g., sky regions due to changing illumination), high-frequency regions (e.g., scene edges), and the presence of transient objects (e.g., pedestrians). EU indicates the model's lack of knowledge, prominently appearing in occluded regions where training supervision is insufficient (e.g., the trees behind and thus occluded by the gate).
• Figure 5: A case where AU dominates EU. The highly reflective surface of the display case in the foreground incurs specular reflections. AU arises due to the presence of data noise caused by the inconsistency of light across different training views.