Neural Radiance Field in Autonomous Driving: A Survey

TL;DR

This survey comprehensively surveys Neural Radiance Field (NeRF) techniques applied to Autonomous Driving (AD) across perception, 3D reconstruction, SLAM, and simulation. It presents a taxonomy of NeRF-AD methods, detailing data augmentation, model-integrated perception, dynamic and surface reconstruction, inverse rendering, multiple SLAM representations, and both image- and LiDAR-based simulations. The work highlights current capabilities, identifies gaps in handling dynamic, outdoor, and large-scale driving scenes, and discusses directions such as temporal consistency, generalizable priors, and efficient rendering for real-time deployment. Overall, the paper aims to be a reference for researchers and industry practitioners seeking to leverage NeRF to advance AD systems.

Abstract

Neural Radiance Field (NeRF) has garnered significant attention from both academia and industry due to its intrinsic advantages, particularly its implicit representation and novel view synthesis capabilities. With the rapid advancements in deep learning, a multitude of methods have emerged to explore the potential applications of NeRF in the domain of Autonomous Driving (AD). However, a conspicuous void is apparent within the current literature. To bridge this gap, this paper conducts a comprehensive survey of NeRF's applications in the context of AD. Our survey is structured to categorize NeRF's applications in Autonomous Driving (AD), specifically encompassing perception, 3D reconstruction, simultaneous localization and mapping (SLAM), and simulation. We delve into in-depth analysis and summarize the findings for each application category, and conclude by providing insights and discussions on future directions in this field. We hope this paper serves as a comprehensive reference for researchers in this domain. To the best of our knowledge, this is the first survey specifically focused on the applications of NeRF in the Autonomous Driving domain.

Figures (14)

• Figure 1: A taxonomy of Neural Radiance Field in Autonomous Driving.
• Figure 2: Overview of NeRF's application in autonomous driving perception: (a) NeRF can be used for data augmentation by reconstructing scenes from either generated data or collected real data. (b) NeRF's implicit representation and neural rendering can be integrated into model training to enhance performance.
• Figure 3: The pipeline of Drive-3DAug3daug.
• Figure 4: The pipeline of S4Chayler2023s4c.
• Figure 5: The pipeline of UniOccpan2023uniocc.