Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spatial-VLN: Zero-Shot Vision-and-Language Navigation With Explicit Spatial Perception and Exploration

Lu Yue, Yue Fan, Shiwei Lian, Yu Zhao, Jiaxin Yu, Liang Xie, Feitian Zhang

TL;DR

This work tackles the lack of fine-grained spatial grounding in zero-shot Vision-and-Language Navigation within continuous environments. It introduces Spatial-VLN, a framework that couples a Spatial Perception Enhancement module with an Explored Multi-expert Reasoning module to ground language into 3D spatial actions while actively exploring to resolve ambiguities. The approach achieves state-of-the-art results in VLN-CE benchmarks and demonstrates strong sim-to-real transfer through value-based waypoint sampling and a DRL-based controller on real robots. These results highlight the practical impact of integrating explicit spatial perception and exploration for robust, generalizable embodied navigation. Future work aims to enhance long-term memory to further improve performance in long-horizon tasks.

Abstract

Zero-shot Vision-and-Language Navigation (VLN) agents leveraging Large Language Models (LLMs) excel in generalization but suffer from insufficient spatial perception. Focusing on complex continuous environments, we categorize key perceptual bottlenecks into three spatial challenges: door interaction,multi-room navigation, and ambiguous instruction execution, where existing methods consistently suffer high failure rates. We present Spatial-VLN, a perception-guided exploration framework designed to overcome these challenges. The framework consists of two main modules. The Spatial Perception Enhancement (SPE) module integrates panoramic filtering with specialized door and region experts to produce spatially coherent, cross-view consistent perceptual representations. Building on this foundation, our Explored Multi-expert Reasoning (EMR) module uses parallel LLM experts to address waypoint-level semantics and region-level spatial transitions. When discrepancies arise between expert predictions, a query-and-explore mechanism is activated, prompting the agent to actively probe critical areas and resolve perceptual ambiguities. Experiments on VLN-CE demonstrate that Spatial VLN achieves state-of-the-art performance using only low-cost LLMs. Furthermore, to validate real-world applicability, we introduce a value-based waypoint sampling strategy that effectively bridges the Sim2Real gap. Extensive real-world evaluations confirm that our framework delivers superior generalization and robustness in complex environments. Our codes and videos are available at https://yueluhhxx.github.io/Spatial-VLN-web/.

Spatial-VLN: Zero-Shot Vision-and-Language Navigation With Explicit Spatial Perception and Exploration

TL;DR

This work tackles the lack of fine-grained spatial grounding in zero-shot Vision-and-Language Navigation within continuous environments. It introduces Spatial-VLN, a framework that couples a Spatial Perception Enhancement module with an Explored Multi-expert Reasoning module to ground language into 3D spatial actions while actively exploring to resolve ambiguities. The approach achieves state-of-the-art results in VLN-CE benchmarks and demonstrates strong sim-to-real transfer through value-based waypoint sampling and a DRL-based controller on real robots. These results highlight the practical impact of integrating explicit spatial perception and exploration for robust, generalizable embodied navigation. Future work aims to enhance long-term memory to further improve performance in long-horizon tasks.

Abstract

Zero-shot Vision-and-Language Navigation (VLN) agents leveraging Large Language Models (LLMs) excel in generalization but suffer from insufficient spatial perception. Focusing on complex continuous environments, we categorize key perceptual bottlenecks into three spatial challenges: door interaction,multi-room navigation, and ambiguous instruction execution, where existing methods consistently suffer high failure rates. We present Spatial-VLN, a perception-guided exploration framework designed to overcome these challenges. The framework consists of two main modules. The Spatial Perception Enhancement (SPE) module integrates panoramic filtering with specialized door and region experts to produce spatially coherent, cross-view consistent perceptual representations. Building on this foundation, our Explored Multi-expert Reasoning (EMR) module uses parallel LLM experts to address waypoint-level semantics and region-level spatial transitions. When discrepancies arise between expert predictions, a query-and-explore mechanism is activated, prompting the agent to actively probe critical areas and resolve perceptual ambiguities. Experiments on VLN-CE demonstrate that Spatial VLN achieves state-of-the-art performance using only low-cost LLMs. Furthermore, to validate real-world applicability, we introduce a value-based waypoint sampling strategy that effectively bridges the Sim2Real gap. Extensive real-world evaluations confirm that our framework delivers superior generalization and robustness in complex environments. Our codes and videos are available at https://yueluhhxx.github.io/Spatial-VLN-web/.
Paper Structure (15 sections, 6 equations, 5 figures, 5 tables)

This paper contains 15 sections, 6 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional VLN-CE
  4. Zero-shot VLN-CE
  5. Method
  6. Problem Formulation
  7. Spatial-Challenging Tasks
  8. Spatial-VLN
  9. Real-World Transfer Strategy
  10. Experiments
  11. Experimental Setup
  12. Simulation Environmental Results
  13. Real-World Experimental Results
  14. Ablation Studies
  15. Conclusion

Figures (5)

  • Figure 1: Overview of challenging spatial tasks and our proposed Spatial-VLN framework. (a) Three spatial challenging tasks: (1) Doorway Interaction – the agent must reason about door state (open/closed), orientation, and semantic transitions between connected regions; (2) Multi-room Transition – the instruction involves navigating through multiple large or ambiguous regions, requiring recognition of region boundaries and semantic shifts; (3) Landmark-Sparse Navigation – the instruction lacks explicit landmarks, forcing the agent to infer spatial intent based on vague directional cues and minimal semantic guidance. (b) The proposed framework Spatial-VLN contains spatial perception enhancement module and explored multi-expert reasoning module. Our approach is validated using various LLM experts and demonstrates robust transferability to real-world environments.
  • Figure 2: The architecture of the proposed Spatial-VLN framework. The framework comprises two core components: the Spatial Perception Enhancement (SPE) module, which generates spatially consistent semantic descriptions while enriching spatial attributes regarding doors and regional transitions, and the Explored Multi-expert Reasoning (EMR) module, which executes dual-expert reasoning based on the enhanced perception. By aligning waypoint-level and region-level inferences, the EMR module triggers an active exploration mechanism to resolve ambiguities whenever decision inconsistencies arise.
  • Figure 3: Illustration of prompts for various LLM-based experts within Spatial-VLN. The prompts empower specialized experts in task decomposition, progress estimation, perception filtering, and history summarization, as well as those focused on exploration and reasoning, to perform critical sub-tasks from key perception extraction to final action decision-making.
  • Figure 4: Visualization of Spatial-VLN deployed in real-world environments. (a) Configuration of the physical robotic platform. (b) An office navigation sequence involving ambiguous instructions, illustrating the activation of the active exploration mechanism to rectify decision ambiguity. (c) A multi-region navigation task in a home environment involving door interactions, demonstrating how door attribute recognition and regional perception enhancement facilitate complex transitions between distinct areas.
  • Figure 5: Visualization of candidate waypoint distribution in real-world environments. Value-based maps, incorporating free-space availability and semantic richness outperform traditional candidate predictors in real-world scenarios. These value-sampled waypoints provide reliable navigation targets and viewpoints for the dual modules of Spatial-VLN, facilitating effective sim-to-real transfer.