IOVS4NeRF:Incremental Optimal View Selection for Large-Scale NeRFs

TL;DR

This paper tackles the problem of large-scale NeRF reconstructions under limited compute by introducing IOVS4NeRF, an uncertainty-guided incremental view-selection framework. It combines rendering uncertainty, modeled via Gaussian RGBs along rays, with positional uncertainty derived from UAV trajectories using a Voronoi-based information gain, forming a hybrid uncertainty $\psi^2(I)=Norm(\sigma_{pos}^2(I))+Norm(\sigma_{rgb}^2(I))$ to select the most informative next view. The approach integrates 5D image information within an Instant-NGP backbone and utilizes a trajectory-type classifier (planar vs non-planar) and Voronoi constructs to guide data acquisition. Experiments on real-world UAV datasets demonstrate that the hybrid uncertainty-driven strategy achieves high-fidelity reconstructions with substantially reduced data and computational requirements, enabling scalable large-scale NeRF applications.

Abstract

Large-scale Neural Radiance Fields (NeRF) reconstructions are typically hindered by the requirement for extensive image datasets and substantial computational resources. This paper introduces IOVS4NeRF, a framework that employs an uncertainty-guided incremental optimal view selection strategy adaptable to various NeRF implementations. Specifically, by leveraging a hybrid uncertainty model that combines rendering and positional uncertainties, the proposed method calculates the most informative view from among the candidates, thereby enabling incremental optimization of scene reconstruction. Our detailed experiments demonstrate that IOVS4NeRF achieves high-fidelity NeRF reconstruction with minimal computational resources, making it suitable for large-scale scene applications.

Figures (5)

• Figure 1: IOVS4NeRF is a flexible framework that actively expands the existing training set with newly captured samples based on hybrid uncertainties of candidate views.
• Figure 2: Overview of Our Framework. It consists of three parts: I. We train NeRF using 5D image information, modeling color as a Gaussian distribution to compute rendering uncertainty. II. We calculate positional uncertainty from training images using a classifier, combining it with rendering uncertainty to form hybrid uncertainty. III. The image with the highest hybrid uncertainty is iteratively added to the training set until the desired reconstruction quality or image limit is reached.
• Figure 3: (a) We use a Voronoi-based method, where a top-level planner updates nodes and a bottom-level planner refines them based on potential fields. (b) We cluster points based on maximum volume and local density, quantifying uncertainty through importance values and density measures. (c) Planar flight trajectory. (d) Non-planar flight trajectory.
• Figure 4: Rendering effect visualization of different methods (partial experimental results).
• Figure 5: Rendering effect visualization of different components (partial experimental results).