Neural Volume Rendering: NeRF And Beyond

TL;DR

The paper surveys the rapid rise of neural volume rendering, anchored by NeRF, and traces its precursors in neural implicit surfaces as well as a broad annotated bibliography of ensuing methods. It concisely interprets how NeRF and its successors formulate images by integrating density and color predicted by neural fields along rays, while highlighting key improvements in speed, dynamics, relighting, shape, composition, and pose estimation. The authors emphasize both the impressive rendering detail and current limitations (training efficiency, static scenes, lighting, generalization) and suggest potential directions, including returning focus to surface-based representations for scalability. Overall, the work provides a structured overview of the field’s trajectory and a resource-rich map for researchers navigating neural rendering advancements.

Abstract

Besides the COVID-19 pandemic and political upheaval in the US, 2020 was also the year in which neural volume rendering exploded onto the scene, triggered by the impressive NeRF paper by Mildenhall et al. (2020). Both of us have tried to capture this excitement, Frank on a blog post (Dellaert, 2020) and Yen-Chen in a Github collection (Yen-Chen, 2020). This note is an annotated bibliography of the relevant papers, and we posted the associated bibtex file on the repository.