Adaptive Bounding Box Uncertainties via Two-Step Conformal Prediction

TL;DR

This work targets principled uncertainty quantification for multi-object detection by leveraging conformal prediction to produce per-object bounding-box intervals with guaranteed coverage at level $1-\alpha$. It introduces a two-step conformal pipeline that first constructs class-label prediction sets and then builds adaptive, per-coordinate bounding-box intervals, propagating class uncertainty to the box estimates. The method employs ensemble and quantile-based scoring (Box-Std, Box-Ens, Box-CQR) and a max-rank multiple-testing correction to maintain coverage while balancing interval width across object sizes, demonstrated on COCO, Cityscapes, and BDD100k with real-world detectors. The approach yields valid, actionable uncertainty estimates even when the detector misclassifies objects, enabling safer decisions in autonomy and robotics.

Abstract

Quantifying a model's predictive uncertainty is essential for safety-critical applications such as autonomous driving. We consider quantifying such uncertainty for multi-object detection. In particular, we leverage conformal prediction to obtain uncertainty intervals with guaranteed coverage for object bounding boxes. One challenge in doing so is that bounding box predictions are conditioned on the object's class label. Thus, we develop a novel two-step conformal approach that propagates uncertainty in predicted class labels into the uncertainty intervals of bounding boxes. This broadens the validity of our conformal coverage guarantees to include incorrectly classified objects, thus offering more actionable safety assurances. Moreover, we investigate novel ensemble and quantile regression formulations to ensure the bounding box intervals are adaptive to object size, leading to a more balanced coverage. Validating our two-step approach on real-world datasets for 2D bounding box localization, we find that desired coverage levels are satisfied with practically tight predictive uncertainty intervals.

Figures (14)

• Figure 1: Examples of our method for multiple classes on test images. True bounding boxes are in red, two-sided prediction interval regions are shaded in green. Produced uncertainty estimates come with a probabilistic coverage guarantee of the true boxes.
• Figure 2: A diagram of our proposed two-step conformal approach. We compute conformal quantiles for both class labels and box coordinates on calibration data following the CP framework. These are used on the predictions of a 'black-box' object detector for a new test sample to (1) form a conformal label set with guarantee (✓) which informs our box quantile choice, and (2) form a conformal prediction interval for the bounding box with guarantee (✓), providing a reliable predictive uncertainty estimate.
• Figure 3: Top: Empirical coverage levels marginally across all objects (All) and across objects from selected classes for the three bounding box methods (\ref{['sec:methods_box']}) on the BDD100k dataset. Target coverage is achieved both marginally and for individual classes. Bottom: Coverage levels are stratified by object size (Small, Medium, Large), showing that Box-CQR and in particular Box-Ens provide a more balanced empirical coverage across sizes. However, this comes at the cost of slightly larger intervals, as seen when comparing $MPIW$. We also visualize target coverage () and the marginal coverage distribution (). Displayed densities are results obtained over 1000 trials.
• Figure 4: Every combination of conformal label set and bounding box method is evaluated along two axes for COCO (top row), Cityscapes (middle row) and BDD100k (bottom row). On the vertical axis we display efficiency, i.e., mean set size for label sets (left column) and $MPIW$ for box intervals (right column). On the horizontal axis we display empirical coverage levels. We also draw target coverage () and marginal coverage distributions (). In line with \ref{['tab:nominal_guarantees_check']}, approaches employing ClassThr or Full consistently achieve both label and box target coverage, at the cost of larger prediction sets/intervals. Results are averaged across classes and 100 trials.
• Figure 5: Exact marginal coverage distributions for our three considered datasets (COCO, Cityscapes, BDD100k) on the basis of their calibration set sizes. For each distribution we additionally mark the $1\%$ and $99\%$ quantiles, as well as the desired target coverage. Left: For the conformal class label prediction sets on the basis of a target miscoverage rate $\alpha_L=0.01$. Right: For the conformal box prediction intervals on the basis of a target miscoverage rate $\alpha_B=0.1$. Obtained empirical coverage levels across experiments fall within reasonable regions of the derived coverage distributions.