Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FlightGPT: Towards Generalizable and Interpretable UAV Vision-and-Language Navigation with Vision-Language Models

Hengxing Cai, Jinhan Dong, Jingjun Tan, Jingcheng Deng, Sihang Li, Zhifeng Gao, Haidong Wang, Zicheng Su, Agachai Sumalee, Renxin Zhong

TL;DR

FlightGPT tackles UAV VLN by grounding navigation in Vision-Language Models. It introduces a two-stage training pipeline combining supervised fine-tuning with Group Relative Policy Optimization and a Chain-of-Thought reasoning module to enable interpretable decisions. On CityNav, FlightGPT achieves state-of-the-art results and strong generalization, including a 9.22 percentage-point gain in unseen environments over the strongest baseline. The approach demonstrates the practical potential of unified multimodal perception and reasoning for robust, explainable UAV navigation.

Abstract

Unmanned Aerial Vehicle (UAV) Vision-and-Language Navigation (VLN) is vital for applications such as disaster response, logistics delivery, and urban inspection. However, existing methods often struggle with insufficient multimodal fusion, weak generalization, and poor interpretability. To address these challenges, we propose FlightGPT, a novel UAV VLN framework built upon Vision-Language Models (VLMs) with powerful multimodal perception capabilities. We design a two-stage training pipeline: first, Supervised Fine-Tuning (SFT) using high-quality demonstrations to improve initialization and structured reasoning; then, Group Relative Policy Optimization (GRPO) algorithm, guided by a composite reward that considers goal accuracy, reasoning quality, and format compliance, to enhance generalization and adaptability. Furthermore, FlightGPT introduces a Chain-of-Thought (CoT)-based reasoning mechanism to improve decision interpretability. Extensive experiments on the city-scale dataset CityNav demonstrate that FlightGPT achieves state-of-the-art performance across all scenarios, with a 9.22\% higher success rate than the strongest baseline in unseen environments. Our implementation is publicly available.

FlightGPT: Towards Generalizable and Interpretable UAV Vision-and-Language Navigation with Vision-Language Models

TL;DR

FlightGPT tackles UAV VLN by grounding navigation in Vision-Language Models. It introduces a two-stage training pipeline combining supervised fine-tuning with Group Relative Policy Optimization and a Chain-of-Thought reasoning module to enable interpretable decisions. On CityNav, FlightGPT achieves state-of-the-art results and strong generalization, including a 9.22 percentage-point gain in unseen environments over the strongest baseline. The approach demonstrates the practical potential of unified multimodal perception and reasoning for robust, explainable UAV navigation.

Abstract

Unmanned Aerial Vehicle (UAV) Vision-and-Language Navigation (VLN) is vital for applications such as disaster response, logistics delivery, and urban inspection. However, existing methods often struggle with insufficient multimodal fusion, weak generalization, and poor interpretability. To address these challenges, we propose FlightGPT, a novel UAV VLN framework built upon Vision-Language Models (VLMs) with powerful multimodal perception capabilities. We design a two-stage training pipeline: first, Supervised Fine-Tuning (SFT) using high-quality demonstrations to improve initialization and structured reasoning; then, Group Relative Policy Optimization (GRPO) algorithm, guided by a composite reward that considers goal accuracy, reasoning quality, and format compliance, to enhance generalization and adaptability. Furthermore, FlightGPT introduces a Chain-of-Thought (CoT)-based reasoning mechanism to improve decision interpretability. Extensive experiments on the city-scale dataset CityNav demonstrate that FlightGPT achieves state-of-the-art performance across all scenarios, with a 9.22\% higher success rate than the strongest baseline in unseen environments. Our implementation is publicly available.
Paper Structure (37 sections, 2 equations, 3 figures, 4 tables)

This paper contains 37 sections, 2 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work and Motivation
  3. Evolution of UAV Vision-and-Language Navigation
  4. Vision-Language Models for Multimodal Perception in Navigation
  5. Reinforcement Learning for Enhancing Generalization in Navigation
  6. Limitations of Existing Work and Motivation for FlightGPT
  7. Method
  8. Problem Formulation
  9. FlightGPT
  10. System Overview
  11. SFT for Warm-up
  12. Input, Prompt, and Output Design
  13. Data Generation
  14. Training Strategy
  15. RL for Generalization
  16. ...and 22 more sections

Figures (3)

  • Figure 1: Workflow of FlightGPT for UAV VLN. FlightGPT takes multimodal input comprising a semantic map image and a natural language instruction, performs Chain-of-Thought reasoning to infer the target location, which is used for subsequent executable actions.
  • Figure 2: The two-stage training pipeline of FlightGPT. The pipeline consists of a supervised fine-tuning (SFT) stage using CoT-annotated data generated by a powerful VLM, followed by reinforcement learning (RL) with composite rewards, including goal accuracy, intermediate reasoning, and format compliance.
  • Figure 3: Reward (train) and success rate (test) over training steps.