Rapid Exploration for Open-World Navigation with Latent Goal Models

TL;DR

The paper tackles robust, goal-directed navigation in open-world environments by learning a latent goal representation through an information bottleneck and pairing it with a non-parametric topological memory. RECON samples feasible latent goals to drive frontier-based exploration and incrementally builds a topological map to reach user-specified visual goals. In real-world tests across eight outdoor environments, RECON discovers goals up to 80 meters away in under 20 minutes and remains robust to distractors and non-stationary changes, outperforming several baselines. The dataset used for offline training and evaluation is released to support future research in real-world visual navigation. These contributions offer a practical, data-efficient framework for rapid exploration and navigation without explicit geometric maps.

Abstract

We describe a robotic learning system for autonomous exploration and navigation in diverse, open-world environments. At the core of our method is a learned latent variable model of distances and actions, along with a non-parametric topological memory of images. We use an information bottleneck to regularize the learned policy, giving us (i) a compact visual representation of goals, (ii) improved generalization capabilities, and (iii) a mechanism for sampling feasible goals for exploration. Trained on a large offline dataset of prior experience, the model acquires a representation of visual goals that is robust to task-irrelevant distractors. We demonstrate our method on a mobile ground robot in open-world exploration scenarios. Given an image of a goal that is up to 80 meters away, our method leverages its representation to explore and discover the goal in under 20 minutes, even amidst previously-unseen obstacles and weather conditions. Please check out the project website for videos of our experiments and information about the real-world dataset used at https://sites.google.com/view/recon-robot.

Figures (9)

• Figure 1: We demonstrate RECON on a Clearpath Jackal.
• Figure 2: System overview: Given a goal image (a), RECON explores the environment (b) by sampling prospective latent goals and constructing a topological map of images (white dots), operating only on visual observations. After finding the goal (c), RECON can reuse the map to reach arbitrary goals in the environment (red path in (b)). RECON uses data collected from diverse training environments (d) to learn navigational priors that enable it to quickly explore and learn to reach visual goals a variety of unseen environments (e).
• Figure 3: Graphical model of actions and distances
• Figure 4: Visualizing goal-reaching behavior of the system:(left) Example trajectories to goals discovered by RECON in previously unseen environments. (right) Policies learned by the different methods in one such environment. Only RECON and ECR reach the goal successfully, and RECON takes the shorter route.
• Figure 5: Exploring non-stationary environments: The learned representation and topological graph is robust to visual distractors, enabling reliable navigation to the goal under novel obstacles (c--e) and appearance changes (f--h).