Informative Rays Selection for Few-Shot Neural Radiance Fields

TL;DR

KeyNeRF tackles the slow training of Neural Radiance Fields in few-shot regimes by selecting a minimal, yet informative set of input views that cover the scene and maximize baseline diversity, followed by entropy-based sampling of rays within those views. This two-stage approach enables faster convergence without requiring additional modalities or complex losses and is easy to integrate into existing NeRF implementations. Empirical results on Blender and CO3D benchmarks show consistent improvements over state-of-the-art few-shot methods, highlighting practical impact for rapid 3D reconstruction from limited data.

Abstract

Neural Radiance Fields (NeRF) have recently emerged as a powerful method for image-based 3D reconstruction, but the lengthy per-scene optimization limits their practical usage, especially in resource-constrained settings. Existing approaches solve this issue by reducing the number of input views and regularizing the learned volumetric representation with either complex losses or additional inputs from other modalities. In this paper, we present KeyNeRF, a simple yet effective method for training NeRF in few-shot scenarios by focusing on key informative rays. Such rays are first selected at camera level by a view selection algorithm that promotes baseline diversity while guaranteeing scene coverage, then at pixel level by sampling from a probability distribution based on local image entropy. Our approach performs favorably against state-of-the-art methods, while requiring minimal changes to existing NeRF codebases.

Figures (8)

• Figure 1: Illustration of the view selection procedure. The new camera (red, right) has the most diverse baseline with respect to the set of current cameras (blue, left). A proxy geometry of the scene is shown for reference.
• Figure 2: Probability distribution over pixels (right) for an example input image (left), induced by the local entropy of the image when sampling a batch of rays for training NeRF.
• Figure 3: Quantitative comparison between our KeyNeRF and the original NeRF nerf as a function of the number of poses (left) and iterations (right). Lower is better.
• Figure 4: Qualitative comparison between choosing poses at random (top row) and using the proposed algorithm (bottom row) in a very few-shot setting ($K = 8$).
• Figure 5: Qualitative comparison between choosing poses at random (top row) and using the proposed algorithm (bottom row), as a function of the number of poses $K$. Zoom in for a better view.