Conformal Object Detection by Sequential Risk Control

TL;DR

This work introduces Sequential Conformal Risk Control (SeqCRC) to provide finite-sample, distribution-free uncertainty guarantees for Conformal Object Detection (COD), enabling simultaneous control of confidence, localization, and classification losses in a model-agnostic manner. By formalizing three parameters—λ^cnf for confidence, and two dependent parameters for localization and classification—SeqCRC yields statistically valid prediction sets across the full OD pipeline, with joint guarantees on localization and classification risks. The authors define multiple conformal losses and matching schemes, prove theoretical bounds, and present an extensive experimental study on DETR-101 and YOLOv8x using MS-COCO, supported by an open-source COD Toolkit for replication. The results highlight trade-offs between guarantee strength and set informativeness, demonstrate practical applicability across detectors, and provide a scalable framework for safer deployments of OD systems. The COD toolkit and comprehensive benchmarks position COD as a robust, transferable approach for uncertainty-aware object detection in safety-critical applications.

Abstract

Recent advances in object detectors have led to their adoption for industrial uses. However, their deployment in safety-critical applications is hindered by the inherent lack of reliability of neural networks and the complex structure of object detection models. To address these challenges, we turn to Conformal Prediction, a post-hoc predictive uncertainty quantification procedure with statistical guarantees that are valid for any dataset size, without requiring prior knowledge on the model or data distribution. Our contribution is manifold. First, we formally define the problem of Conformal Object Detection (COD). We introduce a novel method, Sequential Conformal Risk Control (SeqCRC), that extends the statistical guarantees of Conformal Risk Control to two sequential tasks with two parameters, as required in the COD setting. Then, we present old and new loss functions and prediction sets suited to applying SeqCRC to different cases and certification requirements. Finally, we present a conformal toolkit for replication and further exploration of our method. Using this toolkit, we perform extensive experiments that validate our approach and emphasize trade-offs and other practical consequences.

Figures (12)

• Figure 1: Conformal Object Detection Pipeline. Our method corrects the output of a given object detector $f$ with finite-sample probabilistic guarantees at a desired error rate $\alpha$. The figure illustrates the key stages: (1) Red bounding boxes show the initial raw predictions from the object detector $f(x)$ which are then filtered by Non-Maximum Suppression to a smaller set. (2) Conformal Confidence Thresholding applies a statistically grounded confidence threshold, resulting in selected predictions shown with yellow bounding boxes with a guaranteed risk. (3) Conformal Localization corrects the previous dashed yellow boxes to build the final statistically valid purple bounding boxes. For each localized object, Conformal Classification constructs a prediction set of labels (e.g., "Set: {Boat, Plane}") from the model's predictions, guaranteed to contain the true class with high probability. These steps together control a user-defined risk in localization and classification at a predetermined level $\alpha^\mathrm{tot}$.
• Figure 2: Abstract diagram representing the lens through which we study object detectors. As seen, we consider the NMS to have already been conducted post-prediction.
• Figure 3: Conformal Localization Step. The localization step, one of the two elements of (3) in Fig. \ref{['fig:first_fig']}, consists in applying corrective margins to each of the predicted bounding boxes.
• Figure 4: Monotonization of the Localization Loss. The plot represents the box-wise recall risk on calibration data as a function of the confidence parameter $\lambda^\mathrm{cnf}$, that is, the box-wise recall loss (Eq. \ref{['eq:od-loss-proportion']}) averaged over the calibration set. The various colors (blue, green, red, cyan, magenta) correspond to different fixed values of $\lambda^\mathrm{loc}$ (0, 125, 250, 375, 500). The solid lines are the raw risks, and the dotted ones are the risks after loss monotonization (cf. Section \ref{['sec:seqcrc-od-algos']}). While we could expect raw losses (and thus raw risks) to be non-increasing, they are usually not, due to (partially) diverging goals between the localization loss and the distance $d$ used in the matching algorithm.
• Figure 5: Conformal Classification Step. The classification step, one of the two elements of (3) in Fig. \ref{['fig:first_fig']}, consists in a predicting a set of labels for each object, with the guarantee to contain the ground truth with high probability.

Theorems & Definitions (9)

• Theorem 1: Theorems 1 and 2 in Angelopoulos_2022_CRC
• Theorem 2
• proof
• Corollary 1
• Remark 1: On the monotonicity assumption
• Remark 2
• proof
• Lemma 1
• proof