Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FloorPlan-VLN: A New Paradigm for Floor Plan Guided Vision-Language Navigation

Kehan Chen, Yan Huang, Dong An, Jiawei He, Yifei Su, Jing Liu, Nianfeng Liu, Liang Wang

Abstract

Existing Vision-Language Navigation (VLN) task requires agents to follow verbose instructions, ignoring some potentially useful global spatial priors, limiting their capability to reason about spatial structures. Although human-readable spatial schematics (e.g., floor plans) are ubiquitous in real-world buildings, current agents lack the cognitive ability to comprehend and utilize them. To bridge this gap, we introduce \textbf{FloorPlan-VLN}, a new paradigm that leverages structured semantic floor plans as global spatial priors to enable navigation with only concise instructions. We first construct the FloorPlan-VLN dataset, which comprises over 10k episodes across 72 scenes. It pairs more than 100 semantically annotated floor plans with Matterport3D-based navigation trajectories and concise instructions that omit step-by-step guidance. Then, we propose a simple yet effective method \textbf{FP-Nav} that uses a dual-view, spatio-temporally aligned video sequence, and auxiliary reasoning tasks to align observations, floor plans, and instructions. When evaluated under this new benchmark, our method significantly outperforms adapted state-of-the-art VLN baselines, achieving more than a 60\% relative improvement in navigation success rate. Furthermore, comprehensive noise modeling and real-world deployments demonstrate the feasibility and robustness of FP-Nav to actuation drift and floor plan distortions. These results validate the effectiveness of floor plan guided navigation and highlight FloorPlan-VLN as a promising step toward more spatially intelligent navigation.

FloorPlan-VLN: A New Paradigm for Floor Plan Guided Vision-Language Navigation

Abstract

Existing Vision-Language Navigation (VLN) task requires agents to follow verbose instructions, ignoring some potentially useful global spatial priors, limiting their capability to reason about spatial structures. Although human-readable spatial schematics (e.g., floor plans) are ubiquitous in real-world buildings, current agents lack the cognitive ability to comprehend and utilize them. To bridge this gap, we introduce \textbf{FloorPlan-VLN}, a new paradigm that leverages structured semantic floor plans as global spatial priors to enable navigation with only concise instructions. We first construct the FloorPlan-VLN dataset, which comprises over 10k episodes across 72 scenes. It pairs more than 100 semantically annotated floor plans with Matterport3D-based navigation trajectories and concise instructions that omit step-by-step guidance. Then, we propose a simple yet effective method \textbf{FP-Nav} that uses a dual-view, spatio-temporally aligned video sequence, and auxiliary reasoning tasks to align observations, floor plans, and instructions. When evaluated under this new benchmark, our method significantly outperforms adapted state-of-the-art VLN baselines, achieving more than a 60\% relative improvement in navigation success rate. Furthermore, comprehensive noise modeling and real-world deployments demonstrate the feasibility and robustness of FP-Nav to actuation drift and floor plan distortions. These results validate the effectiveness of floor plan guided navigation and highlight FloorPlan-VLN as a promising step toward more spatially intelligent navigation.
Paper Structure (24 sections, 6 equations, 15 figures, 9 tables)

This paper contains 24 sections, 6 equations, 15 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Vision-Language Navigation
  4. Hand-Drawn Maps for Navigation
  5. Floor Plan Point Navigation
  6. Task and Dataset
  7. Task Definition
  8. FloorPlan-VLN Dataset Collection
  9. Dataset Analysis
  10. FP-Nav
  11. Spatio-Temporally Aligned Video Stream
  12. Auxiliary Tasks
  13. Unified Training Objective
  14. Modeling Actuation and Floor Plan Noises
  15. Experiment
  16. ...and 9 more sections

Figures (15)

  • Figure 1: Illustration of the FloorPlan-VLN paradigm. Floor plans are derived from Matterport3D scenes, and concise region-based instructions are constructed to train agents to navigate by leveraging floor plans as global spatial priors.
  • Figure 2: The dataset collection pipeline of the FloorPlan-VLN dataset.
  • Figure 3: Hand drawn maps can be converted to standardized representation.
  • Figure 4: Dataset analysis of the FloorPlan-R2R training split.
  • Figure 5: Overview of the FP-Nav framework. The model takes a spatio-temporally aligned dual-view video stream as input, jointly encoding floor plans and egocentric observations. Three auxiliary tasks are designed to strengthen cross-modal alignment, and the model autoregressively predicts navigation actions.
  • ...and 10 more figures