Centerness-based Instance-aware Knowledge Distillation with Task-wise Mutual Lifting for Object Detection on Drone Imagery

TL;DR

This paper presents the first attempt to adapt Knowledge Distillation to object detection on drone imagery and addresses two intrinsic issues: (1) low foreground-background ratio and (2) small instances and complex backgrounds, which lead to inadequate training, resulting insufficient distillation.

Abstract

Developing accurate and efficient detectors for drone imagery is challenging due to the inherent complexity of aerial scenes. While some existing methods aim to achieve high accuracy by utilizing larger models, their computational cost is prohibitive for drones. Recently, Knowledge Distillation (KD) has shown promising potential for maintaining satisfactory accuracy while significantly compressing models in general object detection. Considering the advantages of KD, this paper presents the first attempt to adapt it to object detection on drone imagery and addresses two intrinsic issues: (1) low foreground-background ratio and (2) small instances and complex backgrounds, which lead to inadequate training, resulting insufficient distillation. Therefore, we propose a task-wise Lightweight Mutual Lifting (Light-ML) module with a Centerness-based Instance-aware Distillation (CID) strategy. The Light-ML module mutually harmonizes the classification and localization branches by channel shuffling and convolution, integrating teacher supervision across different tasks during back-propagation, thus facilitating training the student model. The CID strategy extracts valuable regions surrounding instances through the centerness of proposals, enhancing distillation efficacy. Experiments on the VisDrone, UAVDT, and COCO benchmarks demonstrate that the proposed approach promotes the accuracies of existing state-of-the-art KD methods with comparable computational requirements. Codes will be available upon acceptance.

Figures (7)

• Figure 1: Challenges to object detection on drone imagery (VisDrone): (a) low foreground-background ratio and (b) small instances and complex backgrounds.
• Figure 2: Framework of our proposed distillation approach. Lightweight Mutual Lifting (Light-ML) mechanism is integrated into the detection heads of the student model for feature lifting and Centerness-based Instance-aware Distillation (CID) introduces an adaptive knowledge weighting algorithm for focal distillation combined with global distillation, enabling the extraction of additional supervision information from the teacher models. Notably, our proposed method can be applied to existing logit-based distillation approaches easily.
• Figure 3: Visualization of the frequency distribution of the mean (a) IoU, (b) GIoU and (c) DIoU value for each ground truth box in the VisDrone dataset, where small instances refer to instances with an area less than $32\times 32$
• Figure 4: Visualization of the frequency distribution of the area ratio between the positive sample regions of LD and the ground truth boxes in the VisDrone dataset, where: (a) instances with an area less than $32\times 32$, (b) instances with an area not less than $32\times 32$.
• Figure 5: Visualization of VLR regions using CID and LD with different $\gamma_{LD}$ values. Highlighted areas indicate activated regions for distillation.