Offline Visual Representation Learning for Embodied Navigation

TL;DR

The paper addresses learning visual representations for embodied navigation by introducing Offline Visual Representation Learning (OVRL), a two-stage pipeline that first pretrains a visual encoder with self-supervised learning on a large, diverse, pre-rendered indoor-image dataset and then online-finetunes visuomotor representations for specific tasks. Across Gibson, HM3D, and MP3D, OVRL significantly improves ImageNav and ObjectNav performance, with the same pretrained encoder generalizing to unseen scenes and datasets. Extensive ablations show that pretraining data quality and diversity, image augmentations, and finetuning strategies are crucial, while the benefits persist and even grow under long training horizons (up to 2B frames). These results challenge tabula rasa training norms and highlight a practical, scalable approach to improving embodied AI visuomotor capabilities.

Abstract

How should we learn visual representations for embodied agents that must see and move? The status quo is tabula rasa in vivo, i.e. learning visual representations from scratch while also learning to move, potentially augmented with auxiliary tasks (e.g. predicting the action taken between two successive observations). In this paper, we show that an alternative 2-stage strategy is far more effective: (1) offline pretraining of visual representations with self-supervised learning (SSL) using large-scale pre-rendered images of indoor environments (Omnidata), and (2) online finetuning of visuomotor representations on specific tasks with image augmentations under long learning schedules. We call this method Offline Visual Representation Learning (OVRL). We conduct large-scale experiments - on 3 different 3D datasets (Gibson, HM3D, MP3D), 2 tasks (ImageNav, ObjectNav), and 2 policy learning algorithms (RL, IL) - and find that the OVRL representations lead to significant across-the-board improvements in state of art, on ImageNav from 29.2% to 54.2% (+25% absolute, 86% relative) and on ObjectNav from 18.1% to 23.2% (+5.1% absolute, 28% relative). Importantly, both results were achieved by the same visual encoder generalizing to datasets that were not seen during pretraining. While the benefits of pretraining sometimes diminish (or entirely disappear) with long finetuning schedules, we find that OVRL's performance gains continue to increase (not decrease) as the agent is trained for 2 billion frames of experience.

Figures (9)

• Figure 1: OVRL consists of two steps: (1) offline pretraining of the visual representations using large-scale pre-rendered images of indoor environments using DINO caron2021emerging, and (2, 3) downstream finetuning of the visuomotor representations on the ImageNav and ObjectNav task in Habitat.
• Figure 2: Overview of OVRL, consisting of two steps: 1) offline pretraining of the visual representations using large-scale pre-rendered images of indoor environments using DINO 2) downstream finetuning of the visuomotor representations on the ImageNav RL task in Habitat.
• Figure 3: Our policy architecture for ObjectNav task when using a RGBD camera and a GPS+Compass sensor. The RGB encoder is initialized with our pretrained weights.
• Figure 4: ImageNav Success/SPL vs. Steps for Scratch (red) and OVRL (blue) on HM3D training (solid) and Gibson testing (dashed).
• Figure 5: Comparison of OVRL against Scratch in the ImageNav task when trained on HM3D scenes and tested on Gibson test scenes