Small Object Detection for Birds with Swin Transformer

TL;DR

Addresses the difficulty of detecting small, sparse birds in images. Proposes a Swin Transformer-based neck with hierarchical feature fusion, shifted windows, and Up Merging upsampling to preserve small-object information. Shows that smaller window sizes (e.g., 2) improve small-object AP and that the Swin neck improves AP50/ AP75 over CenterNet baselines on drone/bird datasets. Demonstrates practical gains for targeted small-object detection in ecological monitoring or surveillance.

Abstract

Object detection is the task of detecting objects in an image. In this task, the detection of small objects is particularly difficult. Other than the small size, it is also accompanied by difficulties due to blur, occlusion, and so on. Current small object detection methods are tailored to small and dense situations, such as pedestrians in a crowd or far objects in remote sensing scenarios. However, when the target object is small and sparse, there is a lack of objects available for training, making it more difficult to learn effective features. In this paper, we propose a specialized method for detecting a specific category of small objects; birds. Particularly, we improve the features learned by the neck; the sub-network between the backbone and the prediction head, to learn more effective features with a hierarchical design. We employ Swin Transformer to upsample the image features. Moreover, we change the shifted window size for adapting to small objects. Experiments show that the proposed Swin Transformer-based neck combined with CenterNet can lead to good performance by changing the window sizes. We further find that smaller window sizes (default 2) benefit mAPs for small object detection.

Figures (4)

• Figure 1: Illustration of the proposed shifted window self-attention approach in the Swin Transformer blocks in the neck network. In layer l, a default window partitioning scheme is adopted (window size 2), and self-attention is computed inside each window. In the next layer l + 1, the window partitioning is shifted by half of the window size (here, 1) for producing new windows. The birds in the windows of layer l + 1 cross the boundaries of the previous windows of layer l, provided cross window attention shifting with smaller window size for small object detection.
• Figure 2: Architecture of the proposed neck network. Feature C5 from the final stage of the backbone is the input. Three features with different scales, also from the backbone, namely C2, C3, and C4 are merged. The output is used for the final detection.
• Figure 3: Illustration of the Up Merging module of Swin Transformer neck for upsampling features. We use it in three stages for upsampling $2\times$ feature size.
• Figure 4: Mean Average Precision of small objects under 32 pixels (AP$_S$) with different windows sizes 2, 3, and 5 of Swin Transformer neck, from Drone2021 IMR validation set.