ETP-R1: Evolving Topological Planning with Reinforcement Fine-tuning for Vision-Language Navigation in Continuous Environments

TL;DR

ETP-R1 addresses Vision-Language Navigation in Continuous Environments by blending graph-based topological planning with large-scale data pretraining and reinforcement fine-tuning. It introduces training-free Gemini instruction augmentation, joint R2R+RxR pretraining, and a three-stage online pipeline featuring closed-loop reinforcement via GRPO, anchored by a DPFT cross-modal planner. Key innovations include a text-guided graph refinement mechanism and a dual-phase encoder architecture that tightly couple language and topology. Empirically, the method sets new state-of-the-art on both R2R-CE and RxR-CE, validating the effectiveness of scaling data and applying reinforcement fine-tuning to graph-based VLN.

Abstract

Vision-Language Navigation in Continuous Environments (VLN-CE) requires an embodied agent to navigate towards target in continuous environments, following natural language instructions. While current graph-based methods offer an efficient, structured approach by abstracting the environment into a topological map and simplifying the action space to waypoint selection, they lag behind methods based on Large Vision-Language Models (LVLMs) in leveraging large-scale data and advanced training paradigms. In this paper, we try to bridge this gap by introducing ETP-R1, a framework that applies the paradigm of scaling up data and Reinforcement Fine-Tuning (RFT) to a graph-based VLN-CE model. To build a strong foundation, we first construct a high-quality, large-scale pretraining dataset using the Gemini API. This dataset consists of diverse, low-hallucination instructions for topological trajectories, providing rich supervision for our graph-based policy to map language to topological paths. This foundation is further strengthened by unifying data from both R2R and RxR tasks for joint pretraining. Building on this, we introduce a three-stage training paradigm, which culminates in the first application of closed-loop, online RFT to a graph-based VLN-CE model, powered by the Group Relative Policy Optimization (GRPO) algorithm. Extensive experiments demonstrate that our approach is highly effective, establishing new state-of-the-art performance across all major metrics on both the R2R-CE and RxR-CE benchmarks. Our code is available at https://github.com/Cepillar/ETP-R1.

Figures (6)

• Figure 1: Comparison of three approaches' pipelines. Our work employs joint dataset pretraining and uses Gemini to generate new data; leverages efficient graph representations; and applies closed-loop RFT to graph-based VLN models for the first time.
• Figure 2: Overview of our approach. Our work focuses on pretraining and online RFT stages within a three-stage training paradigm.
• Figure 3: Visualization of the Gemini instruction annotation. The original Prevalent speaker annotation result is also included for comparison.
• Figure 4: Illustration of the DPFT framework.
• Figure 5: This episode from R2R-CE Val Unseen showcases our model's ability to follow complex instructions. The agent successfully handles a long trajectory by (1) following the backward command, (2) selecting the correct door from two ambiguous options, and (3) following subsequent directional commands to reach the destination.