SALON: Self-supervised Adaptive Learning for Off-road Navigation

TL;DR

This work tackles off-road autonomous navigation by enabling fast online adaptation of traversability through a perception-action loop that leverages visual foundation models and proprioceptive signals. SALON maps rich visual features into BEV space, uses one-shot costmap augmentation, and applies probabilistic, risk-aware predictions (CVaR) to generate adaptive cost and speed maps. It introduces data-management strategies to maintain diverse experiences and an OOD detector to avoid unsafe terrain, achieving performance comparable to data-intensive baselines while requiring orders of magnitude less human input. Real-world tests on ATV, a wheelchair, and a quadruped demonstrate strong generalization and rapid adaptation, with open-source code facilitating deployment on new robots.

Abstract

Autonomous robot navigation in off-road environments presents a number of challenges due to its lack of structure, making it difficult to handcraft robust heuristics for diverse scenarios. While learned methods using hand labels or self-supervised data improve generalizability, they often require a tremendous amount of data and can be vulnerable to domain shifts. To improve generalization in novel environments, recent works have incorporated adaptation and self-supervision to develop autonomous systems that can learn from their own experiences online. However, current works often rely on significant prior data, for example minutes of human teleoperation data for each terrain type, which is difficult to scale with more environments and robots. To address these limitations, we propose SALON, a perception-action framework for fast adaptation of traversability estimates with minimal human input. SALON rapidly learns online from experience while avoiding out of distribution terrains to produce adaptive and risk-aware cost and speed maps. Within seconds of collected experience, our results demonstrate comparable navigation performance over kilometer-scale courses in diverse off-road terrain as methods trained on 100-1000x more data. We additionally show promising results on significantly different robots in different environments. Our code is available at https://theairlab.org/SALON.

Figures (8)

• Figure 1: We present SALON, a framework for off-road navigation with no prior experience. With one prior hand-label, our system running SALON learns from its own experience in the real-world to predict where and how fast to drive.
• Figure 2: $\texttt{SALON}$ Overview: We learn rapidly from online experience with minimal human input to predict cost maps and speed maps.Visual foundation model (VFM) features in the map space, proprioceptive supervision, and smart data strategies together enable perception that adapts quickly to its environment.
• Figure 3: Our system understands the relationship between velocity and traversability, and will command speeds that match the user's tolerance for experienced cost. Top row: costmaps, conditioned on different speeds increasing from left to right; Bottom row: speedmaps, with a user-set maximum cost threshold increasing from left to right.
• Figure 4: Comparison of our method against baselines. Note our method's ability to distinguish the tree line (green dashed line), trail (white dashed line), and the shattered TV hidden in the bushes (red circle).
• Figure 5: $\texttt{SALON}$'s ability to distinguish fine-grained terrain: Rough gravel in the middle of the trail is high cost.