Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheel

TL;DR

The paper tackles the data bottleneck in Vision-and-Language Navigation by introducing SRDF, a fully automated data flywheel that alternates between a language-instruction generator and a navigator to iteratively create, filter, and refine high-quality instruction–trajectory data without human annotation. Through three flywheel rounds built on the R2R dataset, SRDF yields a substantial data-quality upgrade, enabling a pre-trained navigator that surpasses human performance on R2R SPL and strong transfer to diverse VLN tasks. The approach leverages interleaved image–action trajectory encoding, LoRA-tuned MLLMs, and navigator-based filtering (nDTW and SPL) to ensure alignment between language and navigation, resulting in state-of-the-art results across R2R, RxR-English, CVDN, REVERIE, SOON, and R4R. Overall, SRDF demonstrates a scalable, data-centric path to advancing embodied navigation, reducing reliance on manual annotation while achieving broad generalization across tasks and environments.

Abstract

Creating high-quality data for training robust language-instructed agents is a long-lasting challenge in embodied AI. In this paper, we introduce a Self-Refining Data Flywheel (SRDF) that generates high-quality and large-scale navigational instruction-trajectory pairs by iteratively refining the data pool through the collaboration between two models, the instruction generator and the navigator, without any human-in-the-loop annotation. Specifically, SRDF starts with using a base generator to create an initial data pool for training a base navigator, followed by applying the trained navigator to filter the data pool. This leads to higher-fidelity data to train a better generator, which can, in turn, produce higher-quality data for training the next-round navigator. Such a flywheel establishes a data self-refining process, yielding a continuously improved and highly effective dataset for large-scale language-guided navigation learning. Our experiments demonstrate that after several flywheel rounds, the navigator elevates the performance boundary from 70% to 78% SPL on the classic R2R test set, surpassing human performance (76%) for the first time. Meanwhile, this process results in a superior generator, evidenced by a SPICE increase from 23.5 to 26.2, better than all previous VLN instruction generation methods. Finally, we demonstrate the scalability of our method through increasing environment and instruction diversity, and the generalization ability of our pre-trained navigator across various downstream navigation tasks, surpassing state-of-the-art methods by a large margin in all cases.

Figures (5)

• Figure 1: (a) Our Pipeline: After using the (instruction) generator to label paths for data augmentation in navigator training, we leverage the trained navigator to filter high-quality data to train a better generator, and the improved generator refines the data pool to train a stronger navigator, iteratively running on the flywheel. (b) Comparison with state-of-the-art (SoTA) methods: Our approach significantly outperforms SoTA on all tasks. It also surpasses human performance on R2R and approaches human-level results on RxR-English and CVDN (for other tasks, human performance is not reported in their paper). The R2R result is from the test set, while others are from val unseen.
• Figure 2: (a) Our (instruction) generator refines low-quality data filtered in the previous round via sampling to train the next-round navigator, and greedily generates high-confidence instructions as candidates to train next-round generator. Then the navigator filters the data for further use. (b) In filtering, the navigator re-runs instructions to compute the path-path similarity score (nDTW & SPL). High-fidelity data is kept for training while low-quality data will be refined.
• Figure 3: Interleaved trajectory prompt and expected response for training our instruction generator. The LLM's response is the corresponding instruction of this path.
• Figure 4: Visualization of generated instructions.
• Figure 5: Visualization of generated instructions.