ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints

TL;DR

ViKiNG addresses long-range navigation by marrying an image-based local controller with a planning framework that can use geographic hints without assuming map accuracy. The system builds a non-geometric topological graph and employs a latent-goal model together with a learned heuristic to guide search toward distant goals, via an A*-like planner called ViKiNG-A*. Real-world experiments on a jackal platform demonstrate kilometer-scale navigation in unseen environments, with robust performance even when hints are outdated or incorrect. This work shows that modest geographic information can regularize planning and substantially improve the scalability of vision-based navigation in the wild.

Abstract

Robotic navigation has been approached as a problem of 3D reconstruction and planning, as well as an end-to-end learning problem. However, long-range navigation requires both planning and reasoning about local traversability, as well as being able to utilize general knowledge about global geography, in the form of a roadmap, GPS, or other side information providing important cues. In this work, we propose an approach that integrates learning and planning, and can utilize side information such as schematic roadmaps, satellite maps and GPS coordinates as a planning heuristic, without relying on them being accurate. Our method, ViKiNG, incorporates a local traversability model, which looks at the robot's current camera observation and a potential subgoal to infer how easily that subgoal can be reached, as well as a heuristic model, which looks at overhead maps for hints and attempts to evaluate the appropriateness of these subgoals in order to reach the goal. These models are used by a heuristic planner to identify the best waypoint in order to reach the final destination. Our method performs no explicit geometric reconstruction, utilizing only a topological representation of the environment. Despite having never seen trajectories longer than 80 meters in its training dataset, ViKiNG can leverage its image-based learned controller and goal-directed heuristic to navigate to goals up to 3 kilometers away in previously unseen environments, and exhibit complex behaviors such as probing potential paths and backtracking when they are found to be non-viable. ViKiNG is also robust to unreliable maps and GPS, since the low-level controller ultimately makes decisions based on egocentric image observations, using maps only as planning heuristics. For videos of our experiments, please check out our project page https://sites.google.com/view/viking-release.

Figures (13)

• Figure 1: Kilometer-scale autonomous navigation with ViKiNG: Our learning-based navigation system takes as input the current egocentric image (c), a photograph of the desired destination (b), and an overhead map (which may be a schematic or satellite image) (a) that provides a hint about the surrounding layout. The robot (d) uses learned models trained in other environments to infer a path to the goal (e), combining local traversability estimates with global heuristics derived from the map. This enables ViKiNG to navigate previously unseen environments (e), where a single traversal might involve following roads (f), off-road driving under a canopy (g), and backtracking from dead ends (h).
• Figure 2: Geographic hints used by ViKiNG. We evaluate our method with either satellite images or schematic roadmaps, though the approach could be used with any other information of this form, such as contour maps.
• Figure 3: An overview of our method. ViKiNG uses latent subgoals $z$ proposed by a learned low-level controller, which operates on raw image observations $o_t$, for global planning on a topological graph $\mathcal{T}$ to reach a distant goal $o_G$, indicates by a photograph and an approximate GPS location. A learned heuristic parses the overhead image $c_t$ to bias this search towards the goal.
• Figure 4: The learned models used by ViKiNG. The latent goal model (left) takes in the current image $o_t$. It also takes in either a true waypoint image $o_w$, or samples a latent waypoint $z_t^w \sim r(z_t^w)$ from a prior distribution, and then predicts, its temporal distance from $o_t$ ($d_t^w$), the action to reach it ($a_t^w$), and its approximate GPS offset ($x_t^w$). The heuristic model (right) takes in an overhead image $c_t$, the approximate GPS coordinates of the current location ($x_t$) and destination ($x_G$), and the coordinates of the waypoint inferred by the latent goal model ($x_w$), and predicts an approximate heuristic value of the waypoint $w$ for reaching the final destination.
• Figure 5: Examples of kilometer-scale goal-seeking in previously unseen environments using only egocentric images (right) and a schematic roadmap or satellite image as hints (left). ViKiNG can navigate in complex environments composed of roads, meadows, trees and buildings.