ESOD: Efficient Small Object Detection on High-Resolution Images

TL;DR

This paper tackles the challenge of detecting small objects in high-resolution images without the heavy cost of simply enlarging inputs. It introduces ESOD, a plug-in framework that performs feature-level object-seeking (ObjSeeker), adaptive patch-slicing (AdaSlicer), and sparse detection (SparseHead) within the detector pipeline, making it applicable to both CNN and ViT backbones. The approach leverages a class-agnostic objectness mask and adaptive feature patches to discard background regions, achieving substantial AP gains while reducing GFLOPs, as demonstrated on VisDrone, UAVDT, and TinyPerson. By exploiting high-resolution inputs more efficiently, ESOD enables robust small-object detection in realistic settings with improved speed and reduced memory usage; future work includes further ViT-specific optimizations and broader hardware deployment.

Abstract

Enlarging input images is a straightforward and effective approach to promote small object detection. However, simple image enlargement is significantly expensive on both computations and GPU memory. In fact, small objects are usually sparsely distributed and locally clustered. Therefore, massive feature extraction computations are wasted on the non-target background area of images. Recent works have tried to pick out target-containing regions using an extra network and perform conventional object detection, but the newly introduced computation limits their final performance. In this paper, we propose to reuse the detector's backbone to conduct feature-level object-seeking and patch-slicing, which can avoid redundant feature extraction and reduce the computation cost. Incorporating a sparse detection head, we are able to detect small objects on high-resolution inputs (e.g., 1080P or larger) for superior performance. The resulting Efficient Small Object Detection (ESOD) approach is a generic framework, which can be applied to both CNN- and ViT-based detectors to save the computation and GPU memory costs. Extensive experiments demonstrate the efficacy and efficiency of our method. In particular, our method consistently surpasses the SOTA detectors by a large margin (e.g., 8% gains on AP) on the representative VisDrone, UAVDT, and TinyPerson datasets. Code is available at https://github.com/alibaba/esod.

Figures (12)

• Figure 1: Example from VisDrone zhu2018vision dataset. This image is uniformly sliced into $8~\times~8$ patches. No object exists in most of the patches (masked in gray), while the small objects, e.g., persons, are clustered in one or two patches.
• Figure 2: Architecture of our generic ESOD detector. ObjSeeker is inserted after stem to seek a few regions possibly containing objects (colored grids). AdaSlicer then slices the feature map into small patches, and discards the background regions. SparseHead applies sparse detection on the patches.
• Figure 3: Computation distribution when taking inputs at $1,536\times864$. Though ClusDet yang2019clustered can reduce the computation on the base detector glenn_jocher_2021_4679653, massive extra computation is introduced. Our method avoids this problem.
• Figure 4: Framework comparison between our ESOD (a) and conventional detectors (b). For a generic detector, ObjSeeker is inserted after stem to seek the object-containing regions via estimating a objectness map. AdaSlicer then adaptively slices the feature map into small patches, discards the background regions, and sends the remaining to neck for feature aggregation. SparseHead finally applies sparse detection on head to save further computation.
• Figure 5: Pseudo-labeling strategies to supervise the objectness mask. Our hybrid strategy (c) utilizes Gaussian masks (a) and SAM kirillov2023segment predictions (b).