Data Scaling for Navigation in Unknown Environments

TL;DR

The paper tackles how data quantity and geographic diversity influence real-world generalization of end-to-end visual navigation in unknown environments, using a 4,565-hour crowd-sourced dataset spanning 161 locations. It demonstrates that training with diverse, large-scale geographic data enables robust zero-shot navigation in unseen environments, with performance approaching that of in-domain demonstrations and improvements following a power-law with respect to location diversity (doubling locations yields about a 15% reduction in failures). When data is noisy, simple regression-based imitation learning outperforms more complex generative or sequence-based models, highlighting the importance of data quality and model simplicity in real-world deployment. The work provides practical guidance on data collection strategies and release resources for reproducible evaluation, with broad implications for scaling navigation systems in real robots.

Abstract

Generalization of imitation-learned navigation policies to environments unseen in training remains a major challenge. We address this by conducting the first large-scale study of how data quantity and data diversity affect real-world generalization in end-to-end, map-free visual navigation. Using a curated 4,565-hour crowd-sourced dataset collected across 161 locations in 35 countries, we train policies for point goal navigation and evaluate their closed-loop control performance on sidewalk robots operating in four countries, covering 125 km of autonomous driving. Our results show that large-scale training data enables zero-shot navigation in unknown environments, approaching the performance of policies trained with environment-specific demonstrations. Critically, we find that data diversity is far more important than data quantity. Doubling the number of geographical locations in a training set decreases navigation errors by ~15%, while performance benefit from adding data from existing locations saturates with very little data. We also observe that, with noisy crowd-sourced data, simple regression-based models outperform generative and sequence-based architectures. We release our policies, evaluation setup and example videos on the project page.

Figures (11)

• Figure 1: We present an in-depth investigation of data scaling for end-to-end learned navigation. Real-world experiments demonstrate that generalization to unseen environments scales well with geographical diversity of training data, while benefit from data quantity alone saturates quickly.
• Figure 2: A crowd-sourced navigation episode segmented into 5 demonstrations for policy training.
• Figure 3: The test route segment checkpoints and robot front camera views in the different environments.
• Figure 4: Policies trained with large-scale data achieve higher success rates compared to environment-specific policies. The zero-shot policy almost matches the policy that incorporates both large-scale data and data from the test environments.
• Figure 5: Policy paths until the first failure in Wuhan. See Fig. \ref{['fig:wuhan']} for the mission checkpoints.