Purifying, Labeling, and Utilizing: A High-Quality Pipeline for Small Object Detection

TL;DR

The paper addresses the persistent challenge of small object detection by reframing detection as a holistic pipeline and introducing PLUSNet, which combines three modules—Hierarchical Feature Purifier (HFP), Multiple Criteria Label Assignment (MCLA), and Frequency Decoupled Head (FDHead). HFP purifies low-level features via FFT-based highpass filtering, MCLA expands and balances positive samples using IoU, Position Offset, and Shape criteria, and FDHead tailors classification and regression to low- and high-frequency information, respectively. Across SODA-D, AI-TOD, and MS COCO, PLUSNet yields consistent, significant improvements over strong baselines, validating its plug-and-play applicability and generalization across detectors and scales. While FFT-based purification incurs higher computational cost, the approach demonstrates the value of frequency-domain information in enhancing small object detection and offers clear directions for further optimization and efficiency gains.

Abstract

Small object detection is a broadly investigated research task and is commonly conceptualized as a "pipeline-style" engineering process. In the upstream, images serve as raw materials for processing in the detection pipeline, where pre-trained models are employed to generate initial feature maps. In the midstream, an assigner selects training positive and negative samples. Subsequently, these samples and features are fed into the downstream for classification and regression. Previous small object detection methods often focused on improving isolated stages of the pipeline, thereby neglecting holistic optimization and consequently constraining overall performance gains. To address this issue, we have optimized three key aspects, namely Purifying, Labeling, and Utilizing, in this pipeline, proposing a high-quality Small object detection framework termed PLUSNet. Specifically, PLUSNet comprises three sequential components: the Hierarchical Feature Purifier (HFP) for purifying upstream features, the Multiple Criteria Label Assignment (MCLA) for improving the quality of midstream training samples, and the Frequency Decoupled Head (FDHead) for more effectively exploiting information to accomplish downstream tasks. The proposed PLUS modules are readily integrable into various object detectors, thus enhancing their detection capabilities in multi-scale scenarios. Extensive experiments demonstrate the proposed PLUSNet consistently achieves significant and consistent improvements across multiple datasets for small object detection.

Figures (6)

• Figure 1: Visualization of the lowest-level feature map. The original feature map contains a large amount of noise. After separating high and low frequency information through Fast Fourier Transform nussbaumer1982fast, highpass feature focus on small objects (e.g., traffic lights, vehicles, pedestrians), while lowpass feature concentrates more on overall semantic information (i.e., the street scene).
• Figure 2: Comparison of the single-criterion and multi-criteria label assignment in small object detection. (a) Deviation issue: Even when the proposal is perfectly aligned in size with the ground truth (GT), minor deviations between them can lead to a rapid decrease in the IoU, especially for small objects. Size issue: Despite being fully contained within the GT, small-sized proposals struggle to achieve competitive IoU comparable to larger ones. (b) The distribution of positive sample numbers for mainstream detectors lin2017focalren2015faster and our PLUSNet. The representative MaxIoU methods completely ignore the small-sized objects, while our proposed MCLA assigns samples in a more reasonable manner. (Experimental details are available in Sec. \ref{['exp_se']}.)
• Figure 3: Components of our proposed PLUSNet. We present the lowest-level feature in Hierarchical Feature Purifier, and it can be observed that the features of small objects (traffic-light, traffic-sign, etc.) become more distinct.
• Figure 4: Proportion analysis of simulation experiment. The percentages in the pie chart show the proportion of positive sample for different scales. According to the definition of the SODA-D dataset cheng2023towards, small objects are categorized into Extremely Small (0 to 12 pixels), Relatively Small (12 to 20 pixels), and Generally Small (20 to 32 pixels).
• Figure 5: Visual comparison between the baseline (Faster RCNN) and PLUSNet on the SODA-D test-set. The first column displays the visualization results of the baseline, while the second column showcases the visualization results of PLUSNet. To facilitate comparison, we have zoomed in on specific regions. It is important to note that the SODA-D dataset applies masking to confine the detection objects within a sufficiently small area. This operation directly removes the pixels of larger objects from the original images, leading to the failure of detecting larger objects, as exemplified by the larger vehicle in the second image.