UAV-Assisted Self-Supervised Terrain Awareness for Off-Road Navigation

TL;DR

This work introduces UAV-assisted self-supervised terrain characterization for off-road navigation, addressing the limitations of ground-view perception by leveraging top-down drone imagery to predict terrain properties. Aerial views enable more accurate prediction of vibration, bumpiness, and power consumption through a patch-based ResNet18 predictor trained with self-supervised labels derived from proprioceptive signals. The approach yields consistent RMSE improvements over ground-based imagery, particularly in vegetation-rich environments, and demonstrates real-world applicability via drone-aided scouting and ground-path optimization. The study also analyzes factors limiting onboard perception, such as distance-related blur and pixel density, and provides ablations to understand the improvement mechanisms. Future work proposes incorporating depth information and additional sensors to enhance geometric obstacle detection and extend applicability to diverse terrains, including snow using thermal sensing.

Abstract

Terrain awareness is an essential milestone to enable truly autonomous off-road navigation. Accurately predicting terrain characteristics allows optimizing a vehicle's path against potential hazards. Recent methods use deep neural networks to predict traversability-related terrain properties in a self-supervised manner, relying on proprioception as a training signal. However, onboard cameras are inherently limited by their point-of-view relative to the ground, suffering from occlusions and vanishing pixel density with distance. This paper introduces a novel approach for self-supervised terrain characterization using an aerial perspective from a hovering drone. We capture terrain-aligned images while sampling the environment with a ground vehicle, effectively training a simple predictor for vibrations, bumpiness, and energy consumption. Our dataset includes 2.8 km of off-road data collected in forest environment, comprising 13 484 ground-based images and 12 935 aerial images. Our findings show that drone imagery improves terrain property prediction by 21.37 % on the whole dataset and 37.35 % in high vegetation, compared to ground robot images. We conduct ablation studies to identify the main causes of these performance improvements. We also demonstrate the real-world applicability of our approach by scouting an unseen area with a drone, planning and executing an optimized path on the ground.

Figures (5)

• Figure 1: Visual representation of our experimental setup. We propose a novel approach for self-supervised terrain awareness for UGV, leveraging an aerial viewpoint from a hovering UAV. As illustrated, the UAV offers a perspective that is better aligned with the terrain and experiences fewer occlusions.
• Figure 2: Overview of our pipeline for data collection and terrain properties learning. (Top Left) Extraction of image patches along the robot's path from onboard UGV images involves projecting the image into a BEV and extracting patches at specified points of interest. (Bottom Left) For aerial images, the same patches are extracted relative to the robot's position, identified by an Aruco marker. (Middle) Depiction of the terrain predictor, featuring a ResNet architecture coupled with a fully-connected network for the regression of a single terrain-related metric. (Left) Application of proprioceptive measurements to generate labels that correlate with terrain characteristics.
• Figure 3: Ablation study showing the impact of patches' distance to the UGV's camera on RMSE of prediction of $M_z$. The quantitative results are complemented with typical ground images at different distances, where each row is for a different terrain patch seen from below by the vehicle. The same patch acquired from the UAV is shown on the left, highlighting the absence of deformation compared to ground images.
• Figure 4: Impact of blurring aerial patches on RMSE of prediction of $M_z$. The X-axis is a simulated distance based on the GSD increasing quadratically with the distance. To simulate the observed effects on ground images, we increased the blurring kernel's standard deviation in the Y-axis.
• Figure 5: Field demonstration highlighting our method capability to efficiently estimate the ground properties from a drone survey. A terrain map and an optimized trajectory is calculated for each developed metric. The baseline is the shortest feasible path from the start point to the end point. ① Two obstacles were avoided by all three metrics. ② Detection of a large hole by $M_z$ and $M_{\omega}$.