Neural Visibility Field for Uncertainty-Driven Active Mapping

TL;DR

The key insight is that regions not visible in the training views lead to inherently unreliable color predictions by NeRF at this region, resulting in increased uncertainty in the synthesized views, so NVF nat-urally assigns higher uncertainty to unobserved regions, aiding robots to select the most informative next viewpoints.

Abstract

This paper presents Neural Visibility Field (NVF), a novel uncertainty quantification method for Neural Radiance Fields (NeRF) applied to active mapping. Our key insight is that regions not visible in the training views lead to inherently unreliable color predictions by NeRF at this region, resulting in increased uncertainty in the synthesized views. To address this, we propose to use Bayesian Networks to composite position-based field uncertainty into ray-based uncertainty in camera observations. Consequently, NVF naturally assigns higher uncertainty to unobserved regions, aiding robots to select the most informative next viewpoints. Extensive evaluations show that NVF excels not only in uncertainty quantification but also in scene reconstruction for active mapping, outperforming existing methods.

Figures (9)

• Figure 1: Neural Visibility Field (NVF) is an uncertainty estimation framework for NeRF that accounts for visibility: whether a region is covered by the training views of a NeRF. Visible regions should have low uncertainty (bottom row), and unobserved should have high uncertainty (top row). In this paper, we show that many existing methods in NeRF uncertainty quantification can be viewed as special cases of our framework, and NVF outperforms them empirically in uncertainty quantification and active mapping tasks.
• Figure 2: Active Mapping with NVF. Starting with a small set of initial views, a trained NVF is used to quantify uncertainties among sampled candidate views and chooses the view with maximum uncertainty as the next view to be observed by the agent.
• Figure 3: Qualitative results of entropy estimation: NVF assigns a higher entropy to previously unobserved regions while the baselines do not distinguish between the observed (View 1) and unobserved regions (View 2/3). Schematic illustrations of the poses of View 1, 2, and 3 can be found in supp. material. Note that within each method and scene, all rendered views share the same color bar.
• Figure 4: Reconstruction results and camera view distribution: NVF demonstrates superior reconstruction and scene coverage across all datasets in comparison to baselines. For room scene, only comparable baselines are presented, full results are provided in supp. material.
• Figure 5: NVF Architecture: The MLP block consists of fully connected layers that use the ReLU activation function. The numbers inside the block denote the size of the layer. The final output from the visibility $(v)$ MLP and RGB $(\mu_c)$ MLP are passed through the sigmoid activation function while the RGB Variance $(Q_c)$ MLP uses softplus activation