SCLNet: A Scale-Robust Complementary Learning Network for Object Detection in UAV Images

TL;DR

This work tackles scale challenges in UAV object detection, notably scale variation and small-object robustness, by proposing SCLNet, a scale-robust complementary learning network. It combines two explicit modules—comprehensive-scale complementary learning (CSCL) and inter-scale contrastive complementary learning (ICCL)—with an end-to-end cooperation (ECoop) to form a unified detector. CSCL uses a scale-complementary decoder and a scale-ground-truth–guided loss, while ICCL transfers rich texture information from large objects to small ones via a contrastive network and distillation-like loss, and ECoop fuses these signals into a Cascade RCNN backbone. Experiments on VisDrone and UAVDT show improved multi-scale detection, especially for small objects, with competitive accuracy and practical inference speed, demonstrating the method’s potential for real-world UAV surveillance and autonomous operation. The approach provides explicit modelling of scale challenges and leverages cross-scale information, which can meaningfully impact UAV perception pipelines and related applications.

Abstract

Most recent UAV (Unmanned Aerial Vehicle) detectors focus primarily on general challenge such as uneven distribution and occlusion. However, the neglect of scale challenges, which encompass scale variation and small objects, continues to hinder object detection in UAV images. Although existing works propose solutions, they are implicitly modeled and have redundant steps, so detection performance remains limited. And one specific work addressing the above scale challenges can help improve the performance of UAV image detectors. Compared to natural scenes, scale challenges in UAV images happen with problems of limited perception in comprehensive scales and poor robustness to small objects. We found that complementary learning is beneficial for the detection model to address the scale challenges. Therefore, the paper introduces it to form our scale-robust complementary learning network (SCLNet) in conjunction with the object detection model. The SCLNet consists of two implementations and a cooperation method. In detail, one implementation is based on our proposed scale-complementary decoder and scale-complementary loss function to explicitly extract complementary information as complement, named comprehensive-scale complementary learning (CSCL). Another implementation is based on our proposed contrastive complement network and contrastive complement loss function to explicitly guide the learning of small objects with the rich texture detail information of the large objects, named inter-scale contrastive complementary learning (ICCL). In addition, an end-to-end cooperation (ECoop) between two implementations and with the detection model is proposed to exploit each potential.

Figures (9)

• Figure 1: (a) Scale variation in UAV image. There is a huge difference in scale between larger and smaller objects of the same category of objects in the UAV image. (b) Comparison of image proportion with scale variation $>$ 2x in different scene datasets. The image proportion with scale variation is much higher in the UAV scene than in other scenes.
• Figure 2: Comparison of detection paradigms. (a) The generic detection paradigm. The detection head decodes the representation features extracted by the image encoder to output detection results, the detection performance of which is determined by the quality of the representation features. (b) Our proposed SCLNet. Complementary learning is introduced as a complement to form a comprehensive and robust representation.
• Figure 3: The framework of our SCLNet, consists of two main complementary learning implementations. One is comprehensive-scale complementary learning, which extracts scale-complementary outputs as complements. The another is intra-category contrastive complementary learning, which utilizes the large objects to guide the learning of the small objects. $M$ and $N$ denote the number of multi-scale feature maps and proposals respectively.
• Figure 4: The detailed structure of scale-complementary decoder. The multi-scale features of the image encoder are sequentially used as inputs. The convolutions with different scale kernels, pixel shuffle and multi-scale fusion are performed then. Finally, the multi-scale deformable self-attention is performed to get the final output.
• Figure 5: The specifics of the contrastive complement network, which is parallel to the classification network, and the feature blocks are assigned by ground category, and the larger objects complement the smaller object's feature blocks in the same category. The final output guides the classification network during the training phase.