Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigation

Haoyu Tong, Xiangyu Dong, Xiaoguang Ma, Haoran Zhao, Yaoming Zhou, Chenghao Lin

TL;DR

A Visual-Spatial Reasoning enhanced framework for aerial VLN is proposed, designed to leverage structured visual prompting, enabling Vision-Language Models (VLMs) to perform direct reasoning on image planes without the need for additional training or complex intermediate representations.

Abstract

Existing aerial Vision-Language Navigation (VLN) methods predominantly adopt a detection-and-planning pipeline, which converts open-vocabulary detections into discrete textual scene graphs. These approaches are plagued by inadequate spatial reasoning capabilities and inherent linguistic ambiguities. To address these bottlenecks, we propose a Visual-Spatial Reasoning (ViSA) enhanced framework for aerial VLN. Specifically, a triple-phase collaborative architecture is designed to leverage structured visual prompting, enabling Vision-Language Models (VLMs) to perform direct reasoning on image planes without the need for additional training or complex intermediate representations. Comprehensive evaluations on the CityNav benchmark demonstrate that the ViSA-enhanced VLN achieves a 70.3\% improvement in success rate compared to the fully trained state-of-the-art (SOTA) method, elucidating its great potential as a backbone for aerial VLN systems.

ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigation

TL;DR

A Visual-Spatial Reasoning enhanced framework for aerial VLN is proposed, designed to leverage structured visual prompting, enabling Vision-Language Models (VLMs) to perform direct reasoning on image planes without the need for additional training or complex intermediate representations.

Abstract

Existing aerial Vision-Language Navigation (VLN) methods predominantly adopt a detection-and-planning pipeline, which converts open-vocabulary detections into discrete textual scene graphs. These approaches are plagued by inadequate spatial reasoning capabilities and inherent linguistic ambiguities. To address these bottlenecks, we propose a Visual-Spatial Reasoning (ViSA) enhanced framework for aerial VLN. Specifically, a triple-phase collaborative architecture is designed to leverage structured visual prompting, enabling Vision-Language Models (VLMs) to perform direct reasoning on image planes without the need for additional training or complex intermediate representations. Comprehensive evaluations on the CityNav benchmark demonstrate that the ViSA-enhanced VLN achieves a 70.3\% improvement in success rate compared to the fully trained state-of-the-art (SOTA) method, elucidating its great potential as a backbone for aerial VLN systems.
Paper Structure (21 sections, 5 equations, 2 figures, 3 tables)

This paper contains 21 sections, 5 equations, 2 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Aerial Vision-Language Navigation (VLN)
  4. Visual Prompting
  5. METHODOLOGY
  6. Problem Formulation and Framework Overview
  7. Landmark-Based Waypoint Generation
  8. Perception Phase
  9. Verification Phase
  10. Execution Phase
  11. Experiment
  12. Experiment Setting
  13. Dataset Preparation
  14. Evaluation Metrics
  15. Implementation Details
  16. ...and 6 more sections

Figures (2)

  • Figure 1: Overall architecture of ViSA. The system iterates through three phases per time step: (1) VPG transforms raw observations into SoM-annotated visual representations; (2) VM performs Three-Stage Verification reasoning and emits task primitives; (3) Executor translates semantic decisions into UAV motion. Closed-loop feedback from VM to VPG enables adaptive detection across iterations.
  • Figure 2: Visualization of the VLM's explicit reasoning trace during a navigation episode in the CityNav environment.ViSA demonstrates robust spatial reasoning by accommodating a flawed spatial preposition ("underneath" instead of "on"). The sequence illustrates the logical rejection of an initial false-positive red car ① located in front of the Tram Depot ③, followed by closed-loop guided search that successfully locates and verifies the correct target positioned behind the depot.