Long Range Navigator (LRN): Extending robot planning horizons beyond metric maps

TL;DR

This work makes a key observation that long-range navigation only necessitates identifying good frontier directions for planning instead of full map knowledge, and proposes Long Range Navigator, that learns an intermediate affordance representation mapping high-dimensional camera images to affordable frontiers for planning, and then optimizing for maximum alignment with the desired goal.

Abstract

A robot navigating an outdoor environment with no prior knowledge of the space must rely on its local sensing to perceive its surroundings and plan. This can come in the form of a local metric map or local policy with some fixed horizon. Beyond that, there is a fog of unknown space marked with some fixed cost. A limited planning horizon can often result in myopic decisions leading the robot off course or worse, into very difficult terrain. Ideally, we would like the robot to have full knowledge that can be orders of magnitude larger than a local cost map. In practice, this is intractable due to sparse sensing information and often computationally expensive. In this work, we make a key observation that long-range navigation only necessitates identifying good frontier directions for planning instead of full map knowledge. To this end, we propose Long Range Navigator (LRN), that learns an intermediate affordance representation mapping high-dimensional camera images to `affordable' frontiers for planning, and then optimizing for maximum alignment with the desired goal. LRN notably is trained entirely on unlabeled ego-centric videos making it easy to scale and adapt to new platforms. Through extensive off-road experiments on Spot and a Big Vehicle, we find that augmenting existing navigation stacks with LRN reduces human interventions at test-time and leads to faster decision making indicating the relevance of LRN. https://personalrobotics.github.io/lrn

Figures (13)

• Figure 1: LRN Overview. Our approach LRN finds affordable frontiers as intermediate representation for the robot to head towards and selects one near the goal heading. On the right is the local perception (TOP 50m, BOTTOM 8m) where LRN changes the default navigation direction (green) to an affordable one (blue).
• Figure 2: Overview of our approach LRN. LRN is fed with egocentric camera images and a goal heading vector. LRN is composed of the following components, namely, 1) the Affordance Backbone: computes affordable frontiers in the image space as heatmaps agnostic of the goal. These affordance hotspots are then projected into a discrete set of affordable headings for the robot to follow, 2) the Goal Conditioned Head, wherein the affordance scores are multiplied with a discrete gaussian score around the goal and a separate gaussian around the previous prediction (to maintain consistency). The maximum combined score heading (red) is selected. The local system can then use that frontier as a goal for local planning instead of the true goal. This process then repeats as new sensor information comes in.
• Figure 3: LRN's formulation of the long-range navigation problem.LRN learns the value estimate $V(s, g, f)$ using affordability score $A(s,f)$ for each frontier and the cost to goal estimate $D(f, g_t)$.
• Figure 4: Learning Affordances from Unlabeled Videos. The fisheye image \ref{['fig:image_b']} shows the observation at the start of the trajectory and the path taken by the human. Fig. \ref{['fig:image_c']} shows the hotspots computed by the automatic data labeling pipeline. The blue part is the path leading up to the hotspot and is labeled as 0. The end of the trajectory becomes the hotspot with yellow indicating high score of 1 and fades to red with a low score above 0.
• Figure 5: Heatmaps computed by LRN. The two images on the left show examples from the Racer Heavy. Blue is lower confidence and red is high confidence. LRN finds affordable spots between trees and on hill crests. The two images on the right show heatmap examples from the Insta360 camera that is mounted on the Spot robot. These are pictures from the human data collection. LRN correctly puts high score on the two forks in the road and to the side of bushes. For more qualitative results see Appendix \ref{['app:heatmaps']}.