Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Radiance Fields: Past, Present, and Future

Ansh Mittal

TL;DR

This survey traces the evolution of Neural Radiance Fields (NeRFs) from early image-based and differentiable rendering approaches to modern implicit representations that enable continuous 3D scene modeling. It synthesizes foundational principles, historical milestones, key architectures, datasets, loss functions, and evaluation metrics, while surveying a broad spectrum of applications from novel view synthesis to dynamic scenes, relighting, and editing. The paper highlights core technical advances such as Fourier features, multiresolution hash encodings, MPI-based and octree-like structures, and differentiable ray marchings, framing a trajectory toward scalable, real-time, and semantically rich NeRF systems. It also charts future directions including handling large-scale environments, uncertainty, open-world and multi-object scenes, and hardware-software co-design to enable deployment in AR/VR, robotics, and industry. Overall, the work provides a comprehensive reference for researchers entering NeRFs and a roadmap for advancing neural rendering across diverse domains.

Abstract

The various aspects like modeling and interpreting 3D environments and surroundings have enticed humans to progress their research in 3D Computer Vision, Computer Graphics, and Machine Learning. An attempt made by Mildenhall et al in their paper about NeRFs (Neural Radiance Fields) led to a boom in Computer Graphics, Robotics, Computer Vision, and the possible scope of High-Resolution Low Storage Augmented Reality and Virtual Reality-based 3D models have gained traction from res with more than 1000 preprints related to NeRFs published. This paper serves as a bridge for people starting to study these fields by building on the basics of Mathematics, Geometry, Computer Vision, and Computer Graphics to the difficulties encountered in Implicit Representations at the intersection of all these disciplines. This survey provides the history of rendering, Implicit Learning, and NeRFs, the progression of research on NeRFs, and the potential applications and implications of NeRFs in today's world. In doing so, this survey categorizes all the NeRF-related research in terms of the datasets used, objective functions, applications solved, and evaluation criteria for these applications.

Neural Radiance Fields: Past, Present, and Future

TL;DR

This survey traces the evolution of Neural Radiance Fields (NeRFs) from early image-based and differentiable rendering approaches to modern implicit representations that enable continuous 3D scene modeling. It synthesizes foundational principles, historical milestones, key architectures, datasets, loss functions, and evaluation metrics, while surveying a broad spectrum of applications from novel view synthesis to dynamic scenes, relighting, and editing. The paper highlights core technical advances such as Fourier features, multiresolution hash encodings, MPI-based and octree-like structures, and differentiable ray marchings, framing a trajectory toward scalable, real-time, and semantically rich NeRF systems. It also charts future directions including handling large-scale environments, uncertainty, open-world and multi-object scenes, and hardware-software co-design to enable deployment in AR/VR, robotics, and industry. Overall, the work provides a comprehensive reference for researchers entering NeRFs and a roadmap for advancing neural rendering across diverse domains.

Abstract

The various aspects like modeling and interpreting 3D environments and surroundings have enticed humans to progress their research in 3D Computer Vision, Computer Graphics, and Machine Learning. An attempt made by Mildenhall et al in their paper about NeRFs (Neural Radiance Fields) led to a boom in Computer Graphics, Robotics, Computer Vision, and the possible scope of High-Resolution Low Storage Augmented Reality and Virtual Reality-based 3D models have gained traction from res with more than 1000 preprints related to NeRFs published. This paper serves as a bridge for people starting to study these fields by building on the basics of Mathematics, Geometry, Computer Vision, and Computer Graphics to the difficulties encountered in Implicit Representations at the intersection of all these disciplines. This survey provides the history of rendering, Implicit Learning, and NeRFs, the progression of research on NeRFs, and the potential applications and implications of NeRFs in today's world. In doing so, this survey categorizes all the NeRF-related research in terms of the datasets used, objective functions, applications solved, and evaluation criteria for these applications.
Paper Structure (11 sections, 30 equations, 5 figures)

This paper contains 11 sections, 30 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Basic Principles and Concepts
  3. Computer Graphics
  4. History of Neural Rendering and NeRFs
  5. Important Architectures, Models, and Extensions
  6. Supporting Datasets
  7. Objective Function
  8. Evaluation Metrics
  9. Different Explored Applications
  10. Neural Radiance Fields in Future
  11. Conclusion

Figures (5)

  • Figure 3: Traditional Plenoptic Sampling without occlusions chai2000plenoptic in (a) where the double-wedge (in blue) encloses the fourier support of a light field without occlusions. Here, the double-wedge width is used to calculate Nyquist Rate Sampling, that determines by minimum and maximum scene depths $[z_{min}, z_{max}]$ and $K_x$ (max spatial frequency). In (b), the light field is split into $D$ non-overlapping layers and Nyquist Sampling Rate decreases by a factor of $D$ and (c) represents that without occlussions, the full light field spectrum is sum of the spectra from each layer. In LLFF mildenhall2019local extends this to mitigate occlusions as continuous light fields are reconstructed from MPIs. In (d), occlusions expand the Fourier supports to ||gm (where blue represents support without occlusions and purple regions depict the occlusion expanding the fourier support) doubling the Nyquist Sampling Rate. Further, (e) depicts that separate reconstruction of the light field for $D$ layers decrease the Nyquist Sampling Rate by a factor of $D$ whereas (f) depicts the case that full light spectrum cannot be reconstructed by summation over individual layer spectra because their support union in smaller than support union in (a). Hence, the need for alpha compositing individual light layers from back to front in the primal domain to compute the full light field. Figures used from LLFF original paper mildenhall2019local
  • Figure 4: A model schematic representation of SRNs (Scene Representation Networks) by Sitzmann et al sitzmann2019scene
  • Figure 5: A block representation of NVs (Neural Volumes) by Lombardi et al lombardi2019neural
  • Figure 6: A block diagram representation of NPBGs (Neural Point Based Graphics) by Aliev et al aliev2020neural
  • Figure :