Cascade R-CNN: High Quality Object Detection and Instance Segmentation

TL;DR

The paper introduces Cascade R-CNN, a multi-stage detector that trains with progressively higher IoU thresholds to produce higher-quality bounding boxes and detections while mitigating overfitting and proposal-detector mismatch. By cascading both bounding box regression and detection, and applying the same cascade during inference, the approach yields substantial gains over strong baselines on COCO and generalizes across datasets and architectures. The authors extend the cascade to instance segmentation with Cascade Mask R-CNN and demonstrate consistent improvements in detection and segmentation quality, including on VOC, KITTI, CityPersons, and WiderFace. Overall, Cascade R-CNN provides a simple, robust, and broadly applicable strategy for achieving high-quality object detection and segmentation. The work has influenced subsequent research and been reproduced across major frameworks, underscoring its practical impact.

Abstract

In object detection, the intersection over union (IoU) threshold is frequently used to define positives/negatives. The threshold used to train a detector defines its \textit{quality}. While the commonly used threshold of 0.5 leads to noisy (low-quality) detections, detection performance frequently degrades for larger thresholds. This paradox of high-quality detection has two causes: 1) overfitting, due to vanishing positive samples for large thresholds, and 2) inference-time quality mismatch between detector and test hypotheses. A multi-stage object detection architecture, the Cascade R-CNN, composed of a sequence of detectors trained with increasing IoU thresholds, is proposed to address these problems. The detectors are trained sequentially, using the output of a detector as training set for the next. This resampling progressively improves hypotheses quality, guaranteeing a positive training set of equivalent size for all detectors and minimizing overfitting. The same cascade is applied at inference, to eliminate quality mismatches between hypotheses and detectors. An implementation of the Cascade R-CNN without bells or whistles achieves state-of-the-art performance on the COCO dataset, and significantly improves high-quality detection on generic and specific object detection datasets, including VOC, KITTI, CityPerson, and WiderFace. Finally, the Cascade R-CNN is generalized to instance segmentation, with nontrivial improvements over the Mask R-CNN. To facilitate future research, two implementations are made available at \url{https://github.com/zhaoweicai/cascade-rcnn} (Caffe) and \url{https://github.com/zhaoweicai/Detectron-Cascade-RCNN} (Detectron).

Figures (9)

• Figure 1: (a) and (b) detections by object detectors of increasing qualities, and (c) examples of increasing quality.
• Figure 2: Bounding box localization, classification loss and detection performance of object detectors of increasing IoU threshold $u$.
• Figure 3: The architectures of different frameworks. "I" is input image, "conv" backbone convolutions, "pool" region-wise feature extraction, "H" network head, "B" bounding box, and "C" classification. "B0" is proposals in all architectures.
• Figure 4: IoU histograms of training samples of each cascade stage. The distribution of the 1st stage is the RPN output. Shown in red are the percentage of positives for the corresponding IoU threshold.
• Figure 5: Distribution of the distance vector $\Delta$ of (\ref{['equ:delta']}) (without normalization) at different cascade stages. Top: plot of $(\delta_x, \delta_y)$. Bottom: plot of $(\delta_w,\delta_h)$. Red dots are outliers for the increasing IoU thresholds of later stages, and the statistics shown are obtained after outlier removal.