Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One Sample is Enough to Make Conformal Prediction Robust

Soroush H. Zargarbashi, Mohammad Sadegh Akhondzadeh, Aleksandar Bojchevski

TL;DR

This paper addresses the fragility of conformal prediction under worst-case perturbations by introducing Robust CP with One Sample (RCP1), which certifies robustness using a single noise-augmented forward pass and a convex binary certificate. By reframing robustness around the CP procedure itself, the method achieves reliable coverage without extensive Monte Carlo sampling and extends to various smoothing schemes and to conformal risk control. The key contributions include the RCP1 algorithm, generalization to all smoothing shapes/sizes, a smoothing-based conformal risk control extension, and strong empirical results showing small, robust prediction sets with significantly reduced computational cost. This approach enables robust, efficient conformal predictions for large models and diverse tasks, broadening practical applicability.

Abstract

For any black-box model, conformal prediction (CP) returns prediction sets guaranteed to include the true label with high adjustable probability. Robust CP (RCP) extends the guarantee to the worst case noise up to a pre-defined magnitude. For RCP, a well-established approach is to use randomized smoothing since it is applicable to any black-box model and provides smaller sets compared to deterministic methods. However, smoothing-based robustness requires many model forward passes per each input which is computationally expensive. We show that conformal prediction attains some robustness even with a single forward pass on a randomly perturbed input. Using any binary certificate we propose a single sample robust CP (RCP1). Our approach returns robust sets with smaller average set size compared to SOTA methods which use many (e.g. 100) passes per input. Our key insight is to certify the conformal procedure itself rather than individual conformity scores. Our approach is agnostic to the task (classification and regression). We further extend our approach to smoothing-based robust conformal risk control.

One Sample is Enough to Make Conformal Prediction Robust

TL;DR

This paper addresses the fragility of conformal prediction under worst-case perturbations by introducing Robust CP with One Sample (RCP1), which certifies robustness using a single noise-augmented forward pass and a convex binary certificate. By reframing robustness around the CP procedure itself, the method achieves reliable coverage without extensive Monte Carlo sampling and extends to various smoothing schemes and to conformal risk control. The key contributions include the RCP1 algorithm, generalization to all smoothing shapes/sizes, a smoothing-based conformal risk control extension, and strong empirical results showing small, robust prediction sets with significantly reduced computational cost. This approach enables robust, efficient conformal predictions for large models and diverse tasks, broadening practical applicability.

Abstract

For any black-box model, conformal prediction (CP) returns prediction sets guaranteed to include the true label with high adjustable probability. Robust CP (RCP) extends the guarantee to the worst case noise up to a pre-defined magnitude. For RCP, a well-established approach is to use randomized smoothing since it is applicable to any black-box model and provides smaller sets compared to deterministic methods. However, smoothing-based robustness requires many model forward passes per each input which is computationally expensive. We show that conformal prediction attains some robustness even with a single forward pass on a randomly perturbed input. Using any binary certificate we propose a single sample robust CP (RCP1). Our approach returns robust sets with smaller average set size compared to SOTA methods which use many (e.g. 100) passes per input. Our key insight is to certify the conformal procedure itself rather than individual conformity scores. Our approach is agnostic to the task (classification and regression). We further extend our approach to smoothing-based robust conformal risk control.

Paper Structure

This paper contains 17 sections, 5 theorems, 34 equations, 14 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background
  3. RCP1: Robust CP with One Sample
  4. Robust Conformal Sets with Randomized Smoothing of All Shapes and Sizes
  5. Extension to Conformal Risk Control
  6. Experiments
  7. Conclusion
  8. Related Work
  9. Additional Description of Figures
  10. More on Conformal Prediction
  11. Supplementary to Theory
  12. Robust Conformal Prediction Guarantee
  13. Proofs
  14. Choosing the conservative $1 - \alpha'$
  15. Lower and Upper Bounds for All Shapes and Sizes
  16. ...and 2 more sections

Key Result

Proposition 1

Let $q = {\mathbb{Q}}\left(\alpha;s(X_i + E_{i}, Y_i): (X_i, Y_i) \in {\mathcal{D}}_{n}\}\right)$. Given a certified lower bound $\mathrm{c}^\downarrow[\cdot, {\mathcal{B}}]$ as later defined in eq:certificate-pointwise, and ${\mathcal{E}}_{n+1}=\{E_i\}_{i=1}^{n+1}$, for any perturbation $\delta \in

Figures (14)

  • Figure 1: [Left] Coverage of vanilla CP, our robust RCP1, and the SOTA BinCP under adversarial attack (\ref{['sec:fig-manual']}). ${\mathcal{C}}_r$ denotes sets with robustness guarantee up to radius $r$, and $C_0$ is guaranteed only for clean points -- but still using same process only with $r = 0$. Even sets from ${\mathcal{C}}_0$ show significantly higher coverage compared to vanilla due to randomization. [Middle] The average time to compute, and the average set size for both RCP1 and BinCP, with ResNet and ViT models on the ImageNet dataset; both axis are log-scaled and pareto-optimal points are at the lower-left. RCP1 is more efficient. Both plots are with $\sigma=0.5$. [Right] Smoothing-based robust conformal risk control. We show the coverage and miscoverage of the RCP1 mask for the class "car" in the segmentation task. Here risk is set to false negative rate.
  • Figure 2: Illustration of the theory behind RCP1. The probabilities $\beta_i$ never need to be computed, since due to the convexity of $\mathrm{c}^\downarrow$ we can directly work with $1 - \alpha$. Further description in \ref{['sec: Robust CP with a Single Sample']}.
  • Figure 3: [Left] Samples from the $\mathrm{Beta}$ distribution of clean coverage, and worst-case coverage. The dashed line is the empirical average, and the overlapping solid line is $\mathrm{c}^\downarrow[1 - \alpha, {\mathcal{B}}]$. [Middle] The empirical, and theoretical worst-case coverage. [Right] The robust $1 - \alpha'$ for two smoothing schemes.
  • Figure 4: [Left] Proportion, and coverage of the prediction sets with $\le5$, and $\le10$ elements for the ViT model. [Middle] $|{\mathcal{C}}_{r, \mathrm{BinCP}}| - |{\mathcal{C}}_{r, \mathrm{RCP1\xspace}}|$ for the CIFAR-10 dataset with a ResNet model. [Right] ImageNet dataset and ViT models ($r=0.25$). In all plots $\sigma=0.5$ and RCP1 uses a single sample.
  • Figure 4: Empirical coverage and average set size for different radii ($r$), for CIFAR-10 dataset with ResNet model and $\sigma=0.25$.
  • ...and 9 more figures

Theorems & Definitions (9)

  • Proposition 1
  • proof
  • Lemma 1
  • Proposition 2
  • proof : Proof of \ref{['thrm:certificate-convex']}
  • Lemma 2
  • proof
  • Lemma 3
  • proof