SeeNav-Agent: Enhancing Vision-Language Navigation with Visual Prompt and Step-Level Policy Optimization

TL;DR

The paper tackles perception, reasoning, and planning shortcomings in vision-language navigation by introducing a zero-shot dual-view visual prompt to reduce hallucinations and a novel SRGPO post-training algorithm with verifiable process rewards for dense step-level feedback. By converting planning aspects into a VQA-like setup and leveraging random step grouping, SRGPO achieves stable, data-efficient enhancements over prior RFT methods. Experiments on EmbodiedBench demonstrate substantial performance gains, with GPT-4.1+VP reaching 86.7% success rate and VP+SRGPO on Qwen2.5-VL-3B-Ins achieving 72.3%, surpassing previous SOTA models. The approach also shows improved training stability and generalization, including out-of-domain scenarios, highlighting its practical impact for robust embodied navigation through LVLMs.

Abstract

Existing Vision-Language Navigation (VLN) agents based on Large Vision-Language Models (LVLMs) often suffer from perception errors, reasoning errors, and planning errors, which significantly hinder their navigation performance. To address these limitations, a novel VLN agent framework, named SeeNav-Agent, is proposed in this work. First, to reduce perception hallucinations of the visual module of the VLN agent, a dual-view Visual Prompt (VP) technique is introduced in the input space, which can also improve the agent's understanding of current spatial states. Subsequently, a novel step-level Reinforcement Fine-Tuning (RFT) method, Step Reward Group Policy Optimization (SRGPO), is designed for the post-training of VLN agents. In SRGPO, we first define verifiable process rewards for the navigation task, and then perform efficient step-level advantage estimation by randomly grouping different navigation steps. SRGPO provides dense reward signals for the reinforcement learning process of the VLN agent and enhances its planning capability. Experimental results on the EmbodiedBench Navigation benchmark indicate that by introducing the zero-shot VP module, the GPT-4.1 achieves a navigation success rate of 86.7%, surpassing the current best LVLM by approximately 20 percentage points (pp). Through post-training based on SRGPO, the Qwen2.5-VL-3B model reaches a navigation success rate of 72.3%, outperforming the best existing LVLM model by 5.6 pp. Moreover, compared to RFT algorithms such as GRPO and GiGPO, the proposed SRGPO demonstrates significant improvements in training stability, convergence efficiency, and generalization capability.

Figures (6)

• Figure 1: The key motivations.Left: Analysis for different navigation error types from LVLM-based VLN agents. Right: Examples of different error types.
• Figure 2: Overview of SeeNav-Agent. Different from previous VLN works tend to use single view image as input and use method like GRPO or GiGPO for RFT, our SeeNav-Agent designs a dual-view input with visual prompt to enhance the visual module in a zero-shot manner, and proposes SRGPO to introduce process reward signals efficiently during the RFT stage by randomly grouping steps.
• Figure 3: Example of Dual-View Visual Prompt. The left part is the BEV, and the right part is the FV. The numbers 0 to 7 represent different action IDs. Yellow part of the agent marker indicates the left side, purple part indicates the right side, and the green arrow points to the front of the agent.
• Figure 4: Training Process of Qwen2.5-VL-3B-Instruct. The solid line and the shaded region represent the mean and standard deviation obtained from multiple training runs.
• Figure 5: Case Comparison of vanilla-GPT 4.1 and GPT4.1 with VP. The bottom part shows the image and text feedback from the environment after taking actions. This case clearly shows that, with the dual-view VP technique, the visual hallucination can be reduced and the spatial understanding capability of the VLN agent can be enhanced.