Soft-NMS -- Improving Object Detection With One Line of Code

TL;DR

The paper addresses the limitation of greedy NMS, which can discard near-duplicate detections and miss objects due to hard suppression. It proposes Soft-NMS, a simple post-processing modification that decays detection scores as a function of overlap with the top box, using Gaussian or linear weightings, without retraining. Empirically, Soft-NMS yields consistent mAP gains across VOC and COCO and across several detectors, while preserving computational efficiency. This makes Soft-NMS a practical, drop-in improvement for existing object detection pipelines with minimal implementation effort.

Abstract

Non-maximum suppression is an integral part of the object detection pipeline. First, it sorts all detection boxes on the basis of their scores. The detection box M with the maximum score is selected and all other detection boxes with a significant overlap (using a pre-defined threshold) with M are suppressed. This process is recursively applied on the remaining boxes. As per the design of the algorithm, if an object lies within the predefined overlap threshold, it leads to a miss. To this end, we propose Soft-NMS, an algorithm which decays the detection scores of all other objects as a continuous function of their overlap with M. Hence, no object is eliminated in this process. Soft-NMS obtains consistent improvements for the coco-style mAP metric on standard datasets like PASCAL VOC 2007 (1.7% for both R-FCN and Faster-RCNN) and MS-COCO (1.3% for R-FCN and 1.1% for Faster-RCNN) by just changing the NMS algorithm without any additional hyper-parameters. Using Deformable-RFCN, Soft-NMS improves state-of-the-art in object detection from 39.8% to 40.9% with a single model. Further, the computational complexity of Soft-NMS is the same as traditional NMS and hence it can be efficiently implemented. Since Soft-NMS does not require any extra training and is simple to implement, it can be easily integrated into any object detection pipeline. Code for Soft-NMS is publicly available on GitHub (http://bit.ly/2nJLNMu).

Figures (7)

• Figure 1: This image has two confident horse detections (shown in red and green) which have a score of 0.95 and 0.8 respectively. The green detection box has a significant overlap with the red one. Is it better to suppress the green box altogether and assign it a score of 0 or a slightly lower score of 0.4?
• Figure 2: The pseudo code in red is replaced with the one in green in Soft-NMS. We propose to revise the detection scores by scaling them as a linear or Gaussian function of overlap.
• Figure 3: In object detection, first category independent region proposals are generated. These region proposals are then assigned a score for each class label using a classification network and their positions are updated slightly using a regression network. Finally, non-maximum-suppression is applied to obtain detections.
• Figure 4: R-FCN Sensitivity to hyper parameters $\sigma$ (Soft-NMS) and $N_t$ (NMS)
• Figure 5: R-FCN : Precision vs Recall at multiple overlap thresholds $O_t$