Detector-Augmented SAMURAI for Long-Duration Drone Tracking

TL;DR

The paper addresses robust RGB-based drone tracking in urban surveillance by evaluating the foundation-model tracker SAMURAI and introducing a detector-augmented extension that leverages continuous detector cues. The approach fuses SAMURAI with a drone detector via a Prediction Fusion Module, enabling reinitialization and correction to combat drift in long sequences. Key contributions include a systematic assessment on the DUT Anti-UAV dataset, the introduction of a long-duration custom dataset, and substantial robustness gains in challenging scenarios, especially during drone exit-reentry events. The work advances practical drone surveillance by showing how detector-informed memory and prompting can stabilize long-term tracking and provides publicly available data to spur future research efforts.

Abstract

Robust long-term tracking of drone is a critical requirement for modern surveillance systems, given their increasing threat potential. While detector-based approaches typically achieve strong frame-level accuracy, they often suffer from temporal inconsistencies caused by frequent detection dropouts. Despite its practical relevance, research on RGB-based drone tracking is still limited and largely reliant on conventional motion models. Meanwhile, foundation models like SAMURAI have established their effectiveness across other domains, exhibiting strong category-agnostic tracking performance. However, their applicability in drone-specific scenarios has not been investigated yet. Motivated by this gap, we present the first systematic evaluation of SAMURAI's potential for robust drone tracking in urban surveillance settings. Furthermore, we introduce a detector-augmented extension of SAMURAI to mitigate sensitivity to bounding-box initialization and sequence length. Our findings demonstrate that the proposed extension significantly improves robustness in complex urban environments, with pronounced benefits in long-duration sequences - especially under drone exit-re-entry events. The incorporation of detector cues yields consistent gains over SAMURAI's zero-shot performance across datasets and metrics, with success rate improvements of up to +0.393 and FNR reductions of up to -0.475.

Figures (13)

• Figure 1: YOLO-FEDER FusionNet Lenhard:2024_YOLOFEDERLenhard:2025_YOLOFEDER exhibits detection dropouts (yellow), producing temporally inconsistent outputs despite the drone's visibility across frames. In contrast, the detector-augmented SAMURAI maintains stable and consistent detections (magenta).
• Figure 2: Schematic overview of the SAMURAI Yang:2024 tracking pipeline extended by a detector module, specifically YOLO-FEDER FusionNet Lenhard:2024_YOLOFEDERLenhard:2025_YOLOFEDER. YOLO-FEDER FusionNet predictions are fused with mask-derived bounding boxes via a Prediction Fusion Module to refine object localization and improve tracking consistency.
• Figure 3: Limitations of SAMURAI when conditioned only on the first-frame bounding box. Yellow boxes denote YOLO-FEDER FusionNet detections, while magenta boxes indicate SAMURAI predictions. Without detector guidance, SAMURAI seems to drift to irrelevant regions or include background structures, leading to errors that propagate across frames.
• Figure 4: Comparison of GT annotations (yellow) and detector-augmented SAMURAI predictions (magenta). While SAMURAI tracks the drone continuously from its first appearance (rightmost), GT labels (leftmost) often begin later, resulting in incomplete temporal coverage.
• Figure 5: Success rate (top) and precision (bottom) curves across sequences from datasets R1 and R2, comparing SAMURAI with GT initialization, SAMURAI with YOLO-FEDER FusionNet initialization, and the detector-augmented SAMURAI.