AirHunt: Bridging VLM Semantics and Continuous Planning for Efficient Aerial Object Navigation

TL;DR

AirHunt tackles open-set aerial object navigation by decoupling VLM semantics from high-frequency planning through a dual-pathway asynchronous architecture. A 3D value map acts as a persistent semantic memory, while Active Dual-Task Reasoning (ADTR) and Semantic-Geometric Coherent Planning (SGCP) selectively query the VLM and balance semantic guidance with motion efficiency. The approach yields continuous flight, reduced latency bottlenecks, and improved navigation performance in large outdoor environments, validated through high-fidelity simulations and real-world experiments. This work demonstrates practical gains in success rate, accuracy, and flight time, establishing a scalable framework for vision-language–driven aerial search in open-world settings.

Abstract

Recent advances in large Vision-Language Models (VLMs) have provided rich semantic understanding that empowers drones to search for open-set objects via natural language instructions. However, prior systems struggle to integrate VLMs into practical aerial systems due to orders-of-magnitude frequency mismatch between VLM inference and real-time planning, as well as VLMs' limited 3D scene understanding. They also lack a unified mechanism to balance semantic guidance with motion efficiency in large-scale environments. To address these challenges, we present AirHunt, an aerial object navigation system that efficiently locates open-set objects with zero-shot generalization in outdoor environments by seamlessly fusing VLM semantic reasoning with continuous path planning. AirHunt features a dual-pathway asynchronous architecture that establishes a synergistic interface between VLM reasoning and path planning, enabling continuous flight with adaptive semantic guidance that evolves through motion. Moreover, we propose an active dual-task reasoning module that exploits geometric and semantic redundancy to enable selective VLM querying, and a semantic-geometric coherent planning module that dynamically reconciles semantic priorities and motion efficiency in a unified framework, enabling seamless adaptation to environmental heterogeneity. We evaluate AirHunt across diverse object navigation tasks and environments, demonstrating a higher success rate with lower navigation error and reduced flight time compared to state-of-the-art methods. Real-world experiments further validate AirHunt's practical capability in complex and challenging environments. Code and dataset will be made publicly available before publication.

Figures (13)

• Figure 1: A drone performs object navigation in a complex outdoor scenario based on a natural language instruction. (a) Custom-built aerial platform equipped with a LiDAR, an onboard computer, and three cameras. (b) Snapshot of the drone's first-person view during navigation. (c) Bird's-eye view of the experimental scenario showing the flight trajectory (red line). The path starts from the corner of the grassy area (yellow triangle) and ends at the target (red star).
• Figure 2: System overview and pipeline comparison. (a) AirHunt features a dual-pathway asynchronous architecture. The VLM-driven reasoning pathway extracts language-conditioned semantic priors to update a 3D value map asynchronously (Async.), while the high-frequency planning pathway continuously (Cont.) harvests this evolving semantic memory to generate motion trajectories. This design enables both modules to operate at their native frequencies while achieving effective synergy, eliminating mutual blocking. (b) Pipeline Comparison. The Sequential approach requires drones to hover for VLM responses, leading to fragmented flight and low search efficiency. The Naive Asynchronous approach enables movement during VLM inference but suffers from action mismatch, as action commands are generated based on stale observations. In contrast, AirHunt repositions the VLM as a high-level semantic generator and temporally integrates semantic information into a persistent 3D value map, enabling continuous flight with adaptive semantic guidance.
• Figure 3: Illustrations of the Active Keyframe Construction and Asynchronous Dual-Task VLM Reasoning Modules. ADTR constructs a coverage-aware keyframe set and a task-aware keyframe set, then queries the VLM for semantic value inference (Task 1) and related object verification (Task 2), respectively.
• Figure 4: Illustration of Semantic-Geometric Viewpoint Generation. Given the instruction "Fly to the tent on the beach", frontier voxels near the target are assigned higher semantic values (red). The prime viewpoint is selected to maximize information gain, computed as the sum of semantic values over observable frontier voxels, thereby focusing on the target tent.
• Figure 5: Comparison of global planning strategies. (a) The constraint graph selectively enforces precedence only when semantic value differences are significant (e.g., A(0.92)$\rightarrow$C(0.62)), allowing geometric optimization for similar-value frontiers (e.g., A and B). (b) Our solution generates a globally coherent path that prioritizes high-value frontiers without revisits. (c)-(f) Baseline approaches tend to yield revisits of previously searched regions (red dashed boxes) or delay visits to semantically critical regions (A and B).