Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Image Valuation in NeRF-based 3D reconstruction

Grigorios Aris Cheimariotis, Antonis Karakottas, Vangelis Chatzis, Angelos Kanlis, Dimitrios Zarpalas

TL;DR

This paper tackles data valuation for NeRF-based 3D reconstruction from unconstrained image collections by introducing a PSNR-based, per-image contribution score, $DV_{psnr}$. It adapts DVRL-inspired principles to track how individual images affect reconstruction quality during training, aggregating PSNR deltas across epochs to quantify each image's impact. Through experiments on the PhotoTourism dataset, the authors show that selecting training images with high $DV_{psnr}$ can improve held-out PSNR, especially for distant viewpoints, and that the scores are reasonably reproducible across random seeds. The work demonstrates a practical, model-agnostic approach to data valuation in NeRF pipelines, while noting limitations such as reliance on simplified NeRF variants and added computational overhead, as well as potential transferability to other resolutions and datasets.

Abstract

Data valuation and monetization are becoming increasingly important across domains such as eXtended Reality (XR) and digital media. In the context of 3D scene reconstruction from a set of images -- whether casually or professionally captured -- not all inputs contribute equally to the final output. Neural Radiance Fields (NeRFs) enable photorealistic 3D reconstruction of scenes by optimizing a volumetric radiance field given a set of images. However, in-the-wild scenes often include image captures of varying quality, occlusions, and transient objects, resulting in uneven utility across inputs. In this paper we propose a method to quantify the individual contribution of each image to NeRF-based reconstructions of in-the-wild image sets. Contribution is assessed through reconstruction quality metrics based on PSNR and MSE. We validate our approach by removing low-contributing images during training and measuring the resulting impact on reconstruction fidelity.

Image Valuation in NeRF-based 3D reconstruction

TL;DR

This paper tackles data valuation for NeRF-based 3D reconstruction from unconstrained image collections by introducing a PSNR-based, per-image contribution score, . It adapts DVRL-inspired principles to track how individual images affect reconstruction quality during training, aggregating PSNR deltas across epochs to quantify each image's impact. Through experiments on the PhotoTourism dataset, the authors show that selecting training images with high can improve held-out PSNR, especially for distant viewpoints, and that the scores are reasonably reproducible across random seeds. The work demonstrates a practical, model-agnostic approach to data valuation in NeRF pipelines, while noting limitations such as reliance on simplified NeRF variants and added computational overhead, as well as potential transferability to other resolutions and datasets.

Abstract

Data valuation and monetization are becoming increasingly important across domains such as eXtended Reality (XR) and digital media. In the context of 3D scene reconstruction from a set of images -- whether casually or professionally captured -- not all inputs contribute equally to the final output. Neural Radiance Fields (NeRFs) enable photorealistic 3D reconstruction of scenes by optimizing a volumetric radiance field given a set of images. However, in-the-wild scenes often include image captures of varying quality, occlusions, and transient objects, resulting in uneven utility across inputs. In this paper we propose a method to quantify the individual contribution of each image to NeRF-based reconstructions of in-the-wild image sets. Contribution is assessed through reconstruction quality metrics based on PSNR and MSE. We validate our approach by removing low-contributing images during training and measuring the resulting impact on reconstruction fidelity.

Paper Structure

This paper contains 15 sections, 4 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Neural Radiance Fields in the Wild
  4. Data valuation
  5. Method
  6. Training configuration
  7. Identifying fair contribution scores
  8. Contribution scores
  9. PSNR Change and Aggregation for a Training Image
  10. Experiments
  11. Dataset
  12. Reproducibility evaluation with correlation
  13. Impact of Training Set Composition
  14. Discussion
  15. Acknowledgments.

Figures (4)

  • Figure 1: Method overview: During NeRF training, we evaluate a small validation set after each training iteration $i$, recording the PSNR associated with image $I^n$. When image $I^n$ is revisited, we compute the change in PSNR by subtracting the previous value from the current one. These PSNR differences are then aggregated to estimate each image's $DV_{psnr}$.
  • Figure 2: Contribution score correlations on the Brandenburg dataset. The x-axis shows contribution scores from one training run, and the y-axis from a different run with a different seed. Each (x, y) point compares scores for the same image.
  • Figure 3: Rendered images from NeRF trained on $DV_{psnr}$ selections (a),(c),(e),(g) and on NeRF-W training sets (b),(d),(f),(h). PSNR = (a) 19.51, (b) 17.59, (c) 21.3, (d) 20.7,(e) 17.31, (f) 17.03, (g) 24.38, (h) 22.9.
  • Figure 4: NeRF-W NIMA valuation and $DV_{psnr}$ valuation.