WayFASTER: a Self-Supervised Traversability Prediction for Increased Navigation Awareness

TL;DR

WayFASTER tackles robust outdoor navigation by learning traversability from self-supervised experience using RGB-D sequences and pose estimates to predict a BEV traversability map. It fuses depth information as 3D voxels and temporal features, and uses a receding horizon estimator for self-supervised labeling and a temporal, depth-voxel fused network. The BEV map feeds a traversability-aware kino-dynamic MPC, improving obstacle avoidance and predictive awareness, demonstrated against baselines and across platforms. The results show improved accuracy over vision-only and heuristic baselines, robust long-path performance, and easy cross-platform deployment, suggesting practical impact for outdoor robotics.

Abstract

Accurate and robust navigation in unstructured environments requires fusing data from multiple sensors. Such fusion ensures that the robot is better aware of its surroundings, including areas of the environment that are not immediately visible but were visible at a different time. To solve this problem, we propose a method for traversability prediction in challenging outdoor environments using a sequence of RGB and depth images fused with pose estimations. Our method, termed WayFASTER (Waypoints-Free Autonomous System for Traversability with Enhanced Robustness), uses experience data recorded from a receding horizon estimator to train a self-supervised neural network for traversability prediction, eliminating the need for heuristics. Our experiments demonstrate that our method excels at avoiding obstacles, and correctly detects that traversable terrains, such as tall grass, can be navigable. By using a sequence of images, WayFASTER significantly enhances the robot's awareness of its surroundings, enabling it to predict the traversability of terrains that are not immediately visible. This enhanced awareness contributes to better navigation performance in environments where such predictive capabilities are essential.

Figures (8)

• Figure 1: WayFASTER accumulates a sequence of RGB-D images to predict a wide-view traversability map.
• Figure 2: Our network model takes a sequence of RGB and depth images to predict a local traversability map. The depth information is fused explicitly in the form of voxels in the network's latent space. A 3D convolutional block fuses the sequence of concatenations and a final 2D convolutional block outputs the traversability map.
• Figure 3: Our system architecture. A sequence of observations is used to predict a local traversability map. The traversability map is directly used by a model predictive control, that samples the map to get the local system parameters.
• Figure 4: Experiment in a challenging environment with tall grass and sharp turns. The first plot shows our method, WayFASTER. The plot on the middle a navigation using WayFAST gasparino2022wayfast, and on the right, the LiDAR-based navigation.
• Figure 5: Example of temporal prediction. For each observation on the left, we show the traversability map prediction and the model predictive controller rollouts. Note that the tree is not visible at time $t_k$. Due to the temporal fusion, the predicted map still keeps track of past occurrences.