DV-VLN: Dual Verification for Reliable LLM-Based Vision-and-Language Navigation

TL;DR

DV-VLN tackles reliability gaps in LLM-driven vision-language navigation by moving from single-shot decisions to a generate–then–verify paradigm. It trains a lightweight LLaMA-2 backbone to produce a structured three-step chain-of-thought (Prediction–View Match–Action) and employs two inference-time verifiers, TFV and MEV, to re-rank multiple CoT-conditioned action candidates. Across R2R, RxR, and REVERIE, DV-VLN achieves competitive or superior results compared with cross-modal systems and clearly outperforms language-only baselines, with ablations confirming the complementary value of TFV and MEV. The method offers practical benefits in interpretability and deployment efficiency due to in-domain training with minimal parameters and no reliance on heavy cross-modal pretraining.

Abstract

Vision-and-Language Navigation (VLN) requires an embodied agent to navigate in a complex 3D environment according to natural language instructions. Recent progress in large language models (LLMs) has enabled language-driven navigation with improved interpretability. However, most LLM-based agents still rely on single-shot action decisions, where the model must choose one option from noisy, textualized multi-perspective observations. Due to local mismatches and imperfect intermediate reasoning, such decisions can easily deviate from the correct path, leading to error accumulation and reduced reliability in unseen environments. In this paper, we propose DV-VLN, a new VLN framework that follows a generate-then-verify paradigm. DV-VLN first performs parameter-efficient in-domain adaptation of an open-source LLaMA-2 backbone to produce a structured navigational chain-of-thought, and then verifies candidate actions with two complementary channels: True-False Verification (TFV) and Masked-Entity Verification (MEV). DV-VLN selects actions by aggregating verification successes across multiple samples, yielding interpretable scores for reranking. Experiments on R2R, RxR (English subset), and REVERIE show that DV-VLN consistently improves over direct prediction and sampling-only baselines, achieving competitive performance among language-only VLN agents and promising results compared with several cross-modal systems.Code is available at https://github.com/PlumJun/DV-VLN.

Figures (5)

• Figure 1: Comparison between direct action decision and our DV-VLN. Given the same instruction, history, and current observations, the direct LLM agent takes a single-step decision and wrongly selects view B. In contrast, DV-VLN first samples multiple candidate actions (B and C) and then applies dual verification, finally choosing candidate C that best matches the instruction and leads to the correct action.
• Figure 2: Overview of DV-VLN. At timestep $t$, a vision-to-text module converts the panoramic observation $O_t$ into textual observation descriptions $D_t$, which, together with the instruction $I$ and navigation history $H_t$, form the navigation input. The LLM performs sampling decoding to generate $K$ candidate outputs $\{A_k\}$ in a structured navigational chain-of-thought format (Prediction, View Match, Action). DV-VLN then applies dual verification—True-False Verification (TFV) and Masked- Entity Verification (MEV)—to score each candidate via multiple verification samples, and selects the final action $a_t$ by ranking candidates according to their summed verification scores.
• Figure 3: Performance sensitivity of DV-VLN on the R2R Val Unseen Subset. Left: varying the number of sampled candidates $K \in \{1,2,4,6,8\}$. Right: varying the number of verification trials $P \in \{1,2,4,6\}$. We report OSR, SR, and SPL.
• Figure 4: Qualitative success case of DV-VLN on R2R. Compared with single-shot decision making, DV-VLN samples multiple CoT-conditioned candidates and re-ranks them via dual verification (TFV+MEV); the correct action (B) receives a higher verification score than the incorrect candidate (D), leading to a reliable final selection.
• Figure :