Neural Radiance Maps for Extraterrestrial Navigation and Path Planning

Abstract

Autonomous vehicles such as the Mars rovers currently lead the vanguard of surface exploration on extraterrestrial planets and moons. In order to accelerate the pace of exploration and science objectives, it is critical to plan safe and efficient paths for these vehicles. However, current rover autonomy is limited by a lack of global maps which can be easily constructed and stored for onboard re-planning. Recently, Neural Radiance Fields (NeRFs) have been introduced as a detailed 3D scene representation which can be trained from sparse 2D images and efficiently stored. We propose to use NeRFs to construct maps for online use in autonomous navigation, and present a planning framework which leverages the NeRF map to integrate local and global information. Our approach interpolates local cost observations across global regions using kernel ridge regression over terrain features extracted from the NeRF map, allowing the rover to re-route itself around untraversable areas discovered during online operation. We validate our approach in high-fidelity simulation and demonstrate lower cost and higher percentage success rate path planning compared to various baselines.

Figures (11)

• Figure 1: Examples of global plan re-routing due to updates from local observations from the Curiosity rover.
• Figure 2: Overview of the Neural Radiance Field (NeRF) training and novel view synthesis task. Given a set of 2D training images with camera poses, the NeRF trains a neural network to represent the scene. Then, given a target camera pose, the NeRF can synthesize a new image from that camera pose of the scene.
• Figure 3: Inputs and outputs of the NeRF neural network. The network takes a 3D point in the scene and viewing direction of the point as input, and learns to predict the color and volumetric density of that point when viewed from that direction. In order to synthesize an image, a NeRF uses multiple rays (viewing directions), and for each ray, queries points along the ray and accumulates their predicted color and densities to compute an RGB pixel value.
• Figure 4: AutoNav local costmap pipeline. First, a depth image (\ref{['fig:autonav_depth']}) is extracted from stereo RGB images from the rover's onboard cameras (\ref{['fig:autonav_stereo']}). Next, the depth image is projected into 3D to produce a point cloud in local frame (\ref{['fig:autonav_point_cloud']}). Finally, the 3D space is gridded up into a 2D costmap, where the cost of each cell is determined from the gradient (slope) and variance (roughness) of points within.
• Figure 5: Overview of online planning framework. The rover uses a global costmap to plan a global path for navigation. The global costmap is continuously updated through onboard imagery processing, resulting in the creation of a local costmap which reflects the current terrain and obstacles around the rover. The rover effectively integrates the local costmap into the global costmap using a NeRF-based global representation, facilitating the adaptation of the local cost values to multiple regions within the global costmap. The rover then combines the new global path with the local costmap values to determine a local path for execution.