Provably Robust Conformal Prediction with Improved Efficiency

TL;DR

This work addresses the vulnerability of conformal prediction to adversarial perturbations and the associated efficiency drawbacks. It introduces RSCP+, a framework that provides provable robustness guarantees for conformal prediction under bounded adversarial noise, and two practical methods, Post-Training Transformation (PTT) and Robust Conformal Training (RCT), to reduce prediction-set size with minimal overhead. RSCP+ replaces the problematic Monte Carlo-based robustness step with a Hoeffding-based bound on the Monte Carlo score and uses a calibrated threshold to maintain coverage, while PTT and RCT empirically shrink the prediction sets on CIFAR-10/100 and ImageNet without sacrificing robustness. The resulting approach yields substantial empirical gains in efficiency (up to 4.36× on CIFAR-10, 5.46× on CIFAR-100, and 16.9× on ImageNet) and provides a publicly available implementation, making robust conformal prediction more practical for large-scale, real-world scenarios.

Abstract

Conformal prediction is a powerful tool to generate uncertainty sets with guaranteed coverage using any predictive model, under the assumption that the training and test data are i.i.d.. Recently, it has been shown that adversarial examples are able to manipulate conformal methods to construct prediction sets with invalid coverage rates, as the i.i.d. assumption is violated. To address this issue, a recent work, Randomized Smoothed Conformal Prediction (RSCP), was first proposed to certify the robustness of conformal prediction methods to adversarial noise. However, RSCP has two major limitations: (i) its robustness guarantee is flawed when used in practice and (ii) it tends to produce large uncertainty sets. To address these limitations, we first propose a novel framework called RSCP+ to provide provable robustness guarantee in evaluation, which fixes the issues in the original RSCP method. Next, we propose two novel methods, Post-Training Transformation (PTT) and Robust Conformal Training (RCT), to effectively reduce prediction set size with little computation overhead. Experimental results in CIFAR10, CIFAR100, and ImageNet suggest the baseline method only yields trivial predictions including full label set, while our methods could boost the efficiency by up to $4.36\times$, $5.46\times$, and $16.9\times$ respectively and provide practical robustness guarantee. Our codes are available at https://github.com/Trustworthy-ML-Lab/Provably-Robust-Conformal-Prediction.

Figures (10)

• Figure 1: An overview of this work: We address two limitations of RSCP gendler2021adversarially by proposing RSCP+ (\ref{['sec:chanllenge1Guarantee']}) & PTT + RCT (\ref{['sec:challenge2Efficiency']}), which enables the first provable and efficient robust conformal prediction. As we show in the experiments in \ref{['Sec: Exp']}, our proposed method could provide useful robust prediction sets information while the baseline failed.
• Figure 2: Diagram illustrating our RSCP+. (Left) (1) The dotted line shows our target: bound Monte-Carlo estimator score $\hat{S}_{\text{RS}}$ under perturbation; (2) The orange arrow denotes the bound of the randomized smoothed score $S_{\text{RS}}$ under perturbation, given by gendler2021adversarially; (3) The grey arrows denote Hoeffding's inequality connecting randomized smoothed score $S_{\text{RS}}$ and Monte Carlo estimator score $\hat{S}_{\text{RS}}$. The target (1) could be derived by (2) + (3). (Right) RSCP+ algorithm.
• Figure 3: Pipeline of our proposed Robust Conformal Training (RCT) method.
• Figure A.1: Density plot of (non)-conformity scores. (Left) The prediction set of RSCP could be decomposed into two parts: (1) Base part generated by vanilla conformal prediction ($|C(x)|$, or RSCP before threshold inflation) and (2) Set inflation by threshold inflation (conservativeness). (Right) By reshaping the cdf, our method could reduce the second part, even without re-ranking the samples. This also gives an intuitive motivation for why we want to reduce the slope of cdf near the threshold (i.e. reduce the density near the threshold).
• Figure B.1: Comparison of score cdf before(left) and after(right) Sigmoid transformation. The red dotted line denotes the desired coverage $1-\alpha$. The left figure shows the cdf of uniformly distributed score $S$ after the ranking transformation. The right figure shows cdf of $\mathcal{Q}_{\text{sig}}(S)$. We could see that Sigmoid transformation greatly reduced the slope near threshold $\tau$ ($\tau = \Phi^{-1}(1-\alpha)$, corresponding to the intersection of the red line and the blue curve).

Theorems & Definitions (36)

• Theorem 1
• proof
• Remark
• Corollary 2
• proof
• Remark
• Lemma A.1
• proof
• Lemma A.2
• proof