FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information

TL;DR

FisherRF tackles active view selection and uncertainty quantification for Radiance Field models trained from limited viewpoints by directly measuring observed information via Fisher Information. The method derives an $\text{EIG}$-based objective for next-best-view selection, employs a Laplace diagonal approximation for tractable Hessian computations, and extends to batch view selection and active mapping using 3D Gaussian Splatting. It also provides pixel-wise uncertainty estimates by propagating diagonal Hessians through volumetric rendering. Across Blender, Mip-NeRF360, Matterport3D, and Gibson datasets, FisherRF achieves state-of-the-art results in view selection, active mapping, and uncertainty quantification, with efficient CUDA implementations reporting high frame rates. This work enables more informative data acquisition and more accurate reconstructions under view-limited regimes and sets a foundation for extending active radiance-field methods to broader, real-world tasks."

Abstract

This study addresses the challenging problem of active view selection and uncertainty quantification within the domain of Radiance Fields. Neural Radiance Fields (NeRF) have greatly advanced image rendering and reconstruction, but the cost of acquiring images poses the need to select the most informative viewpoints efficiently. Existing approaches depend on modifying the model architecture or hypothetical perturbation field to indirectly approximate the model uncertainty. However, selecting views from indirect approximation does not guarantee optimal information gain for the model. By leveraging Fisher Information, we directly quantify observed information on the parameters of Radiance Fields and select candidate views by maximizing the Expected Information Gain(EIG). Our method achieves state-of-the-art results on multiple tasks, including view selection, active mapping, and uncertainty quantification, demonstrating its potential to advance the field of Radiance Fields.

Figures (6)

• Figure 1: A brief overview of our method Given a Radiance Field that was trained with a limited number of views, our method could find the next best view that could maximize information gain by computing the Fisher Information of the radiance field. We illustrate the Information Gain as a heat map on the viewing sphere and show four of the candidate views. Our model can quantify pixel-wise uncertainty, visualized at the bottom left, by examining the Fisher Information on the related parameters of the ray. Our algorithm can be also adapted into an active mapping system, as showcased at the right, that could actively explore and reconstruct the environment.
• Figure 2: An Illustration of Our Active Mapping System Given RGBD captures, we first reconstruct the environment using 3D Gaussians as representation. Afterward, we select a set of goal points from map frontiers and plan the shortest path to each goal. The EIG is computed for each path, and we choose the one with the highest EIG to continue exploration.
• Figure 3: Qualitative Study of our method on Mip360 Dataset From the top to bottom are results from ActiveNeRF, random baseline, our method, and the ground truth. All the models in this figure are implemented on top of 3D Gaussian Splatting kerbl3Dgaussians for better performance on this challenging dataset. We could see baseline models exhibited artifacts in some renderings due to their lack of constraints from nearby training views.
• Figure 4: Qualitative Results on Blender Dataset with 20 and 10 Training Views. All the methods are implemented on 3D Gaussian Splatting and compared in the same training configuration except for different training views selected by different methods.
• Figure 5: Qualitative Comparisons on Active Mapping We compare our method against the ground truth mesh and Active Neural Mapping. Our method exhibited better details and coverage. Our method can successfully produce detailed and high-fidelity reconstruction for the rooms compared to the previous method.