Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FreeFly-Thinking : Aligning Chain-of-Thought Reasoning with Continuous UAV Navigation

Jiaxu Zhou, Shaobo Wang, Zhiyuan Yang, Zhenjun Yu, Tao Li

TL;DR

FreeFly-thinking is introduced, an end-to-end VLN framework that converts the UAV agent's egocentric images and language instructions into a series of actions, inspired by environment of urban architecture proposed by OpenFly.

Abstract

Vision-Language Navigation aims to enable agents to understand natural language instructions and carry out appropriate navigation actions in real-world environments. Most work focuses on indoor settings, with little research in complex outdoor scenes. Current UAV Vision-and-Language Navigation models typically act as black boxes without explicit reasoning. We introduce FreeFly-thinking, an end-to-end VLN framework that converts the UAV agent's egocentric images and language instructions into a series of actions, inspired by environment of urban architecture proposed by OpenFly. We first construct a UAV dataset for navigation task, and then performing natural language chain of thought. We adopt a two-stage training strategy: Supervised fine-tuning and Reinforcement fine-tuning. Experiments on unseen test demonstrate a strong performance, presenting robustness and efficiency in UAV navigation issue.

FreeFly-Thinking : Aligning Chain-of-Thought Reasoning with Continuous UAV Navigation

TL;DR

FreeFly-thinking is introduced, an end-to-end VLN framework that converts the UAV agent's egocentric images and language instructions into a series of actions, inspired by environment of urban architecture proposed by OpenFly.

Abstract

Vision-Language Navigation aims to enable agents to understand natural language instructions and carry out appropriate navigation actions in real-world environments. Most work focuses on indoor settings, with little research in complex outdoor scenes. Current UAV Vision-and-Language Navigation models typically act as black boxes without explicit reasoning. We introduce FreeFly-thinking, an end-to-end VLN framework that converts the UAV agent's egocentric images and language instructions into a series of actions, inspired by environment of urban architecture proposed by OpenFly. We first construct a UAV dataset for navigation task, and then performing natural language chain of thought. We adopt a two-stage training strategy: Supervised fine-tuning and Reinforcement fine-tuning. Experiments on unseen test demonstrate a strong performance, presenting robustness and efficiency in UAV navigation issue.
Paper Structure (14 sections, 4 equations, 3 figures, 2 tables)

This paper contains 14 sections, 4 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Vision-and-Language Navigation
  4. Reinforcement Learning and Reasoning
  5. Methodology
  6. Overview
  7. VLA model for UAV navigation
  8. Dataset and Data collection
  9. Two-Stage Training
  10. Experiments
  11. Experimental details
  12. Evaluation performence
  13. Ablation experiments
  14. Conclusion

Figures (3)

  • Figure 1: Overview for our work, illustrating the complete pipeline from automated Chain-of-Thought (CoT) data preparation based on visual observations and instructions, to the two-stage training paradigm (SFT and RFT) that optimizes the dual-head Freefly-thinking model for UAV navigation.
  • Figure 2: Architecture for Freefly model, which extracts shared multimodal representations from sequential visual observations and language instructions to simultaneously predict explicit Chain-of-Thought reasoning via a Language Model head and continuous 3D spatial waypoints via a Waypoint head
  • Figure 3: Statistical overview of our constructed UAV navigation dataset. (a) The sample size distribution of primary flight maneuvers (e.g., straight, turn). (b) The distribution of physical trajectory lengths. (c) A word cloud visualization highlighting the most frequent terms in the generated Chain-of-Thought (CoT) rationales.