UAV-CodeAgents: Scalable UAV Mission Planning via Multi-Agent ReAct and Vision-Language Reasoning

TL;DR

UAV-CodeAgents presents a scalable, vision-language–guided framework that enables autonomous UAV mission generation through a multi-agent ReAct-inspired architecture. By grounding natural language instructions to pixel-level targets on aerial imagery and enabling iterative plan refinement among heterogeneous agents, the system bridges high-level reasoning with low-level geospatial execution. Experimental results show robust performance, with a 93% success rate and 96.96 s average mission completion at a low decoding temperature, and strong spatial grounding from fine-tuned Qwen2.5VL-7B on 9k satellite images. The approach offers a reusable, extensible platform for real-time UAV planning in dynamic environments and sets the stage for future swarm-based deployments and real-world sensor integration.

Abstract

We present UAV-CodeAgents, a scalable multi-agent framework for autonomous UAV mission generation, built on large language and vision-language models (LLMs/VLMs). The system leverages the ReAct (Reason + Act) paradigm to interpret satellite imagery, ground high-level natural language instructions, and collaboratively generate UAV trajectories with minimal human supervision. A core component is a vision-grounded, pixel-pointing mechanism that enables precise localization of semantic targets on aerial maps. To support real-time adaptability, we introduce a reactive thinking loop, allowing agents to iteratively reflect on observations, revise mission goals, and coordinate dynamically in evolving environments. UAV-CodeAgents is evaluated on large-scale mission scenarios involving industrial and environmental fire detection. Our results show that a lower decoding temperature (0.5) yields higher planning reliability and reduced execution time, with an average mission creation time of 96.96 seconds and a success rate of 93%. We further fine-tune Qwen2.5VL-7B on 9,000 annotated satellite images, achieving strong spatial grounding across diverse visual categories. To foster reproducibility and future research, we will release the full codebase and a novel benchmark dataset for vision-language-based UAV planning.

Figures (5)

• Figure 1: The pipeline of the UAV-CodeAgents system.
• Figure 2: The samples in the benchmark.
• Figure 3: The scheme of simulation.
• Figure 4: Task completion times across temperature settings.
• Figure 5: Mean distance in the pixel-pointing task.