Cascade R-CNN: Delving into High Quality Object Detection

TL;DR

The paper tackles high-quality object detection by introducing Cascade R-CNN, a multi-stage detector architecture that trains detectors with progressively higher IoU thresholds to reduce close false positives and overfitting. It combines cascaded bounding box regression and cascaded detection, using a resampling-based training and inference scheme so that each stage operates on appropriately refined hypotheses. Empirical results on COCO show consistent improvements across baseline detectors (Faster-RCNN, R-FCN, FPN) with modest computational overhead, demonstrating strong generalization and practical impact. The approach achieves state-of-the-art single-model performance and provides a generally applicable framework for building high-precision detectors across architectures, with code to be released for public use.

Abstract

In object detection, an intersection over union (IoU) threshold is required to define positives and negatives. An object detector, trained with low IoU threshold, e.g. 0.5, usually produces noisy detections. However, detection performance tends to degrade with increasing the IoU thresholds. Two main factors are responsible for this: 1) overfitting during training, due to exponentially vanishing positive samples, and 2) inference-time mismatch between the IoUs for which the detector is optimal and those of the input hypotheses. A multi-stage object detection architecture, the Cascade R-CNN, is proposed to address these problems. It consists of a sequence of detectors trained with increasing IoU thresholds, to be sequentially more selective against close false positives. The detectors are trained stage by stage, leveraging the observation that the output of a detector is a good distribution for training the next higher quality detector. The resampling of progressively improved hypotheses guarantees that all detectors have a positive set of examples of equivalent size, reducing the overfitting problem. The same cascade procedure is applied at inference, enabling a closer match between the hypotheses and the detector quality of each stage. A simple implementation of the Cascade R-CNN is shown to surpass all single-model object detectors on the challenging COCO dataset. Experiments also show that the Cascade R-CNN is widely applicable across detector architectures, achieving consistent gains independently of the baseline detector strength. The code will be made available at https://github.com/zhaoweicai/cascade-rcnn.

Figures (7)

• Figure 1: The detection outputs, localization and detection performance of object detectors of increasing IoU threshold $u$.
• Figure 2: Sequential $\Delta$ distribution (without normalization) at different cascade stage. Red dots are outliers when using increasing IoU thresholds, and the statistics are obtained after outlier removal.
• Figure 3: The architectures of different frameworks. "I" is input image, "conv" backbone convolutions, "pool" region-wise feature extraction, "H" network head, "B" bounding box, and "C" classification. "B0" is proposals in all architectures.
• Figure 4: The IoU histogram of training samples. The distribution at 1st stage is the output of RPN. The red numbers are the positive percentage higher than the corresponding IoU threshold.
• Figure 5: (a) is detection performance of individually trained detectors, with their own proposals (solid curves) or Cascade R-CNN stage proposals (dashed curves), and (b) is by adding ground truth to the proposal set.