Dynamic Semantic-Aware Correlation Modeling for UAV Tracking

TL;DR

The paper introduces DSATrack, a dynamic semantic-aware UAV tracking framework that embeds a Dynamic Semantic Relevance Generator into a one-stream Transformer to produce semantic-aware correlation maps for robust target localization under UAV-specific challenges. A Hierarchical Contribution Ranking Pruning strategy selectively removes low-importance Transformer blocks to balance precision and real-time performance, enabling multiple variants suitable for diverse onboard resources. Extensive experiments across UAV benchmarks demonstrate state-of-the-art accuracy, with notable gains from DSAM over baseline and static semantic models, and real-world tests on edge devices show feasible deployment. The work advances UAV perception by explicitly optimizing semantic correlations and providing flexible, hardware-aware model configurations, while acknowledging limitations like lack of re-detection and proposing future work on global tracking capabilities.

Abstract

UAV tracking can be widely applied in scenarios such as disaster rescue, environmental monitoring, and logistics transportation. However, existing UAV tracking methods predominantly emphasize speed and lack exploration in semantic awareness, which hinders the search region from extracting accurate localization information from the template. The limitation results in suboptimal performance under typical UAV tracking challenges such as camera motion, fast motion, and low resolution, etc. To address this issue, we propose a dynamic semantic aware correlation modeling tracking framework. The core of our framework is a Dynamic Semantic Relevance Generator, which, in combination with the correlation map from the Transformer, explore semantic relevance. The approach enhances the search region's ability to extract important information from the template, improving accuracy and robustness under the aforementioned challenges. Additionally, to enhance the tracking speed, we design a pruning method for the proposed framework. Therefore, we present multiple model variants that achieve trade-offs between speed and accuracy, enabling flexible deployment according to the available computational resources. Experimental results validate the effectiveness of our method, achieving competitive performance on multiple UAV tracking datasets. The code is available at https://github.com/zxyyxzz/DSATrack.

Figures (9)

• Figure 1: Comparison with SOTA methods under typical UAV challenges. The first to third rows represent camera motion, fast motion, and low resolution, respectively.
• Figure 2: The framework of DSATrack. The figure above illustrates the overall pipeline of DSATrack, while the two figures below provide a detailed explanation of the Dynamic Semantic Relevance Generator and the Hybrid Attention, respectively.
• Figure 3: Comparison of state-of-the-art methods under representative UAV challenges.
• Figure 4: Visualization of correlation map (a) search region. (b) correlation map before the DSAM. (c) correlation map after the DSAM.
• Figure 5: A comparison of speed and accuracy with state-of-the-art methods. The performance is the average across four datasets.