Breaking Smooth-Motion Assumptions: A UAV Benchmark for Multi-Object Tracking in Complex and Adverse Conditions

TL;DR

DynUAV is introduced, a new benchmark for dynamic UAV-perspective MOT, characterized by intense ego-motion and the resulting complex apparent trajectories, and is anticipated that DynUAV will serve as a demanding testbed to spur progress in real-world UAV-perspective MOT.

Abstract

The rapid movements and agile maneuvers of unmanned aerial vehicles (UAVs) induce significant observational challenges for multi-object tracking (MOT). However, existing UAV-perspective MOT benchmarks often lack these complexities, featuring predominantly predictable camera dynamics and linear motion patterns. To address this gap, we introduce DynUAV, a new benchmark for dynamic UAV-perspective MOT, characterized by intense ego-motion and the resulting complex apparent trajectories. The benchmark comprises 42 video sequences with over 1.7 million bounding box annotations, covering vehicles, pedestrians, and specialized industrial categories such as excavators, bulldozers and cranes. Compared to existing benchmarks, DynUAV introduces substantial challenges arising from ego-motion, including drastic scale changes and viewpoint changes, as well as motion blur. Comprehensive evaluations of state-of-the-art trackers on DynUAV reveal their limitations, particularly in managing the intertwined challenges of detection and association under such dynamic conditions, thereby establishing DynUAV as a rigorous benchmark. We anticipate that DynUAV will serve as a demanding testbed to spur progress in real-world UAV-perspective MOT, and we will make all resources available at link.

Figures (5)

• Figure 1: Visualization of scene diversity and complexity in the DynUAV benchmark. Our benchmark encompasses three primary scenarios: campus, city and road, and nighttime. For each scenario, the upper montage depicts a variety of its constituent sequences, while the lower panel illustrates the temporal progression of a single representative sequence, accompanied with its ground-truth annotations.
• Figure 2: Comparison of the mean and variance of IOU across datasets. The horizontal and vertical axes represent the mean and variance of IoU respectively. The bubble radius corresponds to the proportion of non-overlapping bounding boxes in adjacent frames.
• Figure 3: Bar charts comparing the distributions of trajectory discontinuities across different datasets. Each bar chart displays the proportion of target IDs relative to the number of continuous trajectory segments.
• Figure 4: Fine-grained IDSW analysis. IDSW counts per algorithm across test sequences in DynUAV, revealing performance fluctuations in diverse scenarios.
• Figure 5: Per-Sequence IDSW with CMC. A radar plot showing the number of IDSW for each color-coded algorithm across all test sequences.