Video Individual Counting and Tracking from Moving Drones: A Benchmark and Methods

TL;DR

The paper tackles video-level crowd counting and tracking in large-scale dense scenes captured by moving drones. It introduces MovingDroneCrowd++ as the largest moving-drone dataset for this task and presents GD3A, a density-map decomposition method using pixel-level descriptor association with an adaptive dustbin score $,s$, via optimal transport, along with DVTrack, a descriptor voting-based tracker. The approach achieves state-of-the-art performance, reducing counting error by 47.4% and improving tracking by 39.2% over prior methods, while offering robustness and efficiency advantages. This work bridges research and practice by providing a challenging benchmark and practical methods for drone-based crowd analysis with real-world applicability.

Abstract

Counting and tracking dense crowds in large-scale scenes is highly challenging, yet existing methods mainly rely on datasets captured by fixed cameras, which provide limited spatial coverage and are inadequate for large-scale dense crowd analysis. To address this limitation, we propose a flexible solution using moving drones to capture videos and perform video-level crowd counting and tracking of unique pedestrians across entire scenes. We introduce MovingDroneCrowd++, the largest video-level dataset for dense crowd counting and tracking captured by moving drones, covering diverse and complex conditions with varying flight altitudes, camera angles, and illumination. Existing methods fail to achieve satisfactory performance on this dataset. To this end, we propose GD3A (Global Density Map Decomposition via Descriptor Association), a density map-based video individual counting method that avoids explicit localization. GD3A establishes pixel-level correspondences between pedestrian descriptors across consecutive frames via optimal transport with an adaptive dustbin score, enabling the decomposition of global density maps into shared, inflow, and outflow components. Building on this framework, we further introduce DVTrack, which converts descriptor-level matching into instance-level associations through a descriptor voting mechanism for pedestrian tracking. Experimental results show that our methods significantly outperform existing approaches under dense crowds and complex motion, reducing counting error by 47.4 percent and improving tracking performance by 39.2 percent.

Figures (8)

• Figure 1: Comparison between existing crowd analysis datasets and ours. Existing research has predominantly focused on (a) free-viewpoint images captured by handheld cameras, (b) videos captured by fixed surveillance, or (c) hovering drones. Due to the constraints of these data acquisition setups, prior methods can not perform video-level crowd counting and tracking in large-scale, crowded environments. Our method utilizes moving drones to capture videos covered large-scale scenes and achieves accurate and interpretable video-level crowd counting and tracking.
• Figure 2: Exemplars from the MovingDroneCrowd++ dataset. Due to space constraints, only two frames are displayed for each video clip. Each frame is annotated with a bounding box and an identity ID for every pedestrian head. These examples illustrate that the dataset is captured by moving drones in dense crowd environments and exhibits significant diversity in terms of shooting angles, flight altitudes, and illumination conditions.
• Figure 3: Crowd density statistics of the MovingDroneCrowd++ dataset. (a) Histogram of people per frame. (b) Histogram of distinct identities per clip. These density statistics demonstrate the balance of the dataset split.
• Figure 4: Scene attributes statistics of the MovingDroneCrowd++ dataset. (a) Proportion of illumination conditions. (b) Proportion of shooting locations. (c) Duration histogram of the newly added clips. These scene attributes statistics highlight the diversity and challenging nature of the proposed dataset.
• Figure 5: The pipeline of the proposed GD3A. Given two frames $F_t$ and $F_{t+\delta}$, a backbone extracts feature maps $\mathbf{F}_t$ and $\mathbf{F}_{t+\delta}$, which are filtered using global density maps $\hat{\mathbf{D}}^g_t$ and $\hat{\mathbf{D}}^g_{t+\delta}$ predicted by a pre-trained estimator to retain visual descriptors for pedestrian heads. Subsequently, these descriptors are enhanced with positional coordinates and refined by an AGNN for contextual aggregation. Correspondences between descriptors from two frames are established via Optimal Transport with an adaptive dustbin score $s$, predicted by a dustbin score predictor. The global density map of each frame is decomposed into shared density map $\hat{\mathbf{D}}^s_t$ and $\hat{\mathbf{D}}^s_{t+\delta}$ (not visualized) and outflow/inflow density maps $\hat{\mathbf{D}}^o_t$ and $\hat{\mathbf{D}}^{in}_{t+\delta}$.