AD-Det: Boosting Object Detection in UAV Images with Focused Small Objects and Balanced Tail Classes

TL;DR

This work tackles UAV image object detection under challenging scale variation and long-tail class distributions. It introduces AD-Det, a coarse-to-fine framework that combines Adaptive Small Object Enhancement (ASOE), which localizes small-object regions on a high-resolution feature map for targeted fine-grained detection, with Dynamic Class-Balanced Copy-Paste (DCC), which uses a memory bank and BFS-guided placement to balance tail classes through adaptive data augmentation. Empirical results on VisDrone and UAVDT show substantial improvements over state-of-the-art baselines in AP and related metrics, demonstrating strong practical performance with reasonable computation. The approach offers a scalable, plug-in strategy to enhance UAV detection by jointly addressing scale and imbalance without excessive model complexity, facilitating robust real-world deployment.

Abstract

Object detection in Unmanned Aerial Vehicle (UAV) images poses significant challenges due to complex scale variations and class imbalance among objects. Existing methods often address these challenges separately, overlooking the intricate nature of UAV images and the potential synergy between them. In response, this paper proposes AD-Det, a novel framework employing a coherent coarse-to-fine strategy that seamlessly integrates two pivotal components: Adaptive Small Object Enhancement (ASOE) and Dynamic Class-balanced Copy-paste (DCC). ASOE utilizes a high-resolution feature map to identify and cluster regions containing small objects. These regions are subsequently enlarged and processed by a fine-grained detector. On the other hand, DCC conducts object-level resampling by dynamically pasting tail classes around the cluster centers obtained by ASOE, main-taining a dynamic memory bank for each tail class. This approach enables AD-Det to not only extract regions with small objects for precise detection but also dynamically perform reasonable resampling for tail-class objects. Consequently, AD-Det enhances the overall detection performance by addressing the challenges of scale variations and class imbalance in UAV images through a synergistic and adaptive framework. We extensively evaluate our approach on two public datasets, i.e., VisDrone and UAVDT, and demonstrate that AD-Det significantly outperforms existing competitive alternatives. Notably, AD-Det achieves a 37.5% Average Precision (AP) on the VisDrone dataset, surpassing its counterparts by at least 3.1%.

Figures (8)

• Figure S1: (a) Comparison of scale distribution between VisDrone and COCO. (b) Class distribution of VisDrone and UAVDT. (c) Visualization of high-resolution feature map ($P_3$) in GFL li2020generalized for VisDrone. It can be seen that the high-resolution feature map mainly focuses on small objects. The masked areas on the left indicate regions that contain large objects, which are easier to handle and can be ignored in the fine-grained detection stage.
• Figure S2: The framework overview of the proposed AD-Det. The network adopts a coarse-to-fine strategy and integrates two key modules, i.e., an adaptive small object enhancement (ASOE) module for excavating regions containing small objects and a dynamic class-balanced copy--paste (DCC) module for balancing class distribution. The final results are obtained by fusing global results and local results using non-maximum suppression (NMS). The dashed line between ASOE and DCC indicates training-only.
• Figure S3: Detector workflow and corresponding heatmap visualizations at different network layers.
• Figure S4: The process of subregion generation in ASOE.
• Figure S5: Visualization of qualitative comparison between the baseline detector and our proposed method on VisDrone. The areas marked by the dashed boxes indicate that our approach excels in detecting small objects and tail categories.