Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AirNav: A Large-Scale Real-World UAV Vision-and-Language Navigation Dataset with Natural and Diverse Instructions

Hengxing Cai, Yijie Rao, Ligang Huang, Zanyang Zhong, Jinhan Dong, Jingjun Tan, Wenhao Lu, Renxin Zhong

TL;DR

AirNav introduces a large-scale, real-world UAV vision-and-language navigation benchmark (AirNav) built from urban data to enhance realism, natural instruction diversity, and scale. It combines a realistic AirNav dataset with the AirVLN-R1 navigation model, which employs a two-stage SFT–RFT paradigm and a progressive memory mechanism to integrate multimodal inputs and hierarchical subgoals. The benchmark includes 143k samples, 10 personas for instruction diversity, and four dataset splits to rigorously evaluate seen and unseen generalization, with thorough analyses of instruction naturalness and linguistic characteristics. Real-world UAV tests demonstrate the feasibility of sim-to-real transfer, while ablations and reward design studies reveal the critical components that drive robust navigation and generalization in complex urban environments. AirNav provides a practical, scalable platform for advancing UAV VLN research and facilitating real-world deployments with more naturalistic language and robust planning capabilities.

Abstract

Existing Unmanned Aerial Vehicle (UAV) Vision-Language Navigation (VLN) datasets face issues such as dependence on virtual environments, lack of naturalness in instructions, and limited scale. To address these challenges, we propose AirNav, a large-scale UAV VLN benchmark constructed from real urban aerial data, rather than synthetic environments, with natural and diverse instructions. Additionally, we introduce the AirVLN-R1, which combines Supervised Fine-Tuning and Reinforcement Fine-Tuning to enhance performance and generalization. The feasibility of the model is preliminarily evaluated through real-world tests. Our dataset and code are publicly available.

AirNav: A Large-Scale Real-World UAV Vision-and-Language Navigation Dataset with Natural and Diverse Instructions

TL;DR

AirNav introduces a large-scale, real-world UAV vision-and-language navigation benchmark (AirNav) built from urban data to enhance realism, natural instruction diversity, and scale. It combines a realistic AirNav dataset with the AirVLN-R1 navigation model, which employs a two-stage SFT–RFT paradigm and a progressive memory mechanism to integrate multimodal inputs and hierarchical subgoals. The benchmark includes 143k samples, 10 personas for instruction diversity, and four dataset splits to rigorously evaluate seen and unseen generalization, with thorough analyses of instruction naturalness and linguistic characteristics. Real-world UAV tests demonstrate the feasibility of sim-to-real transfer, while ablations and reward design studies reveal the critical components that drive robust navigation and generalization in complex urban environments. AirNav provides a practical, scalable platform for advancing UAV VLN research and facilitating real-world deployments with more naturalistic language and robust planning capabilities.

Abstract

Existing Unmanned Aerial Vehicle (UAV) Vision-Language Navigation (VLN) datasets face issues such as dependence on virtual environments, lack of naturalness in instructions, and limited scale. To address these challenges, we propose AirNav, a large-scale UAV VLN benchmark constructed from real urban aerial data, rather than synthetic environments, with natural and diverse instructions. Additionally, we introduce the AirVLN-R1, which combines Supervised Fine-Tuning and Reinforcement Fine-Tuning to enhance performance and generalization. The feasibility of the model is preliminarily evaluated through real-world tests. Our dataset and code are publicly available.
Paper Structure (81 sections, 7 equations, 4 figures, 9 tables)

This paper contains 81 sections, 7 equations, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. AirNav benchmark
  4. Task Definition
  5. Success Criteria.
  6. Metrics.
  7. Benchmark Construction
  8. Data Sources and Environment
  9. Step 1: Start and Target Selection.
  10. Step 2: Landmark Planning.
  11. Step 3: Trajectory Synthesis.
  12. Step 4: Instruction Generation.
  13. Dataset and Instruction Analysis
  14. Dataset Splits.
  15. Distance Distribution and Task Difficulty.
  16. ...and 66 more sections

Figures (4)

  • Figure 1: Overview of the AirNav Benchmark Construction Pipeline.
  • Figure 2: Dataset Analysis and Instruction Naturalness of AirNav.
  • Figure 3: Overview of the AirVLN-R1 architecture. The model receives multimodal inputs and predicts an action sequence to control the UAV. A two-stage training paradigm is used to enhance performance.
  • Figure 4: Real-World UAV VLN Deployment in Indoor and Outdoor Scenes.