Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Yet Efficient Conformal Prediction Sets

Soroush H. Zargarbashi, Mohammad Sadegh Akhondzadeh, Aleksandar Bojchevski

TL;DR

This work tackles the vulnerability of conformal prediction (CP) to adversarial evasion and calibration-data poisoning by introducing CDF-Aware Sets (CAS). CAS derives provable robustness using randomized smoothing to bound worst-case conformity-score changes under perturbations and leverages the score's CDF to obtain tighter upper bounds, improving efficiency over prior certificates. It extends robustness guarantees to both feature and label poisoning and to discrete, sparse data, while incorporating finite-sample corrections for practical deployment. The calibration-time certificate further reduces computational cost and set sizes, enabling scalable robust CP on large datasets and graphs. Overall, CAS provides provably robust yet efficient prediction sets that preserve CP’s distribution-free guarantees in adversarial environments, with broad applicability across modalities and data types.

Abstract

Conformal prediction (CP) can convert any model's output into prediction sets guaranteed to include the true label with any user-specified probability. However, same as the model itself, CP is vulnerable to adversarial test examples (evasion) and perturbed calibration data (poisoning). We derive provably robust sets by bounding the worst-case change in conformity scores. Our tighter bounds lead to more efficient sets. We cover both continuous and discrete (sparse) data and our guarantees work both for evasion and poisoning attacks (on both features and labels).

Robust Yet Efficient Conformal Prediction Sets

TL;DR

This work tackles the vulnerability of conformal prediction (CP) to adversarial evasion and calibration-data poisoning by introducing CDF-Aware Sets (CAS). CAS derives provable robustness using randomized smoothing to bound worst-case conformity-score changes under perturbations and leverages the score's CDF to obtain tighter upper bounds, improving efficiency over prior certificates. It extends robustness guarantees to both feature and label poisoning and to discrete, sparse data, while incorporating finite-sample corrections for practical deployment. The calibration-time certificate further reduces computational cost and set sizes, enabling scalable robust CP on large datasets and graphs. Overall, CAS provides provably robust yet efficient prediction sets that preserve CP’s distribution-free guarantees in adversarial environments, with broad applicability across modalities and data types.

Abstract

Conformal prediction (CP) can convert any model's output into prediction sets guaranteed to include the true label with any user-specified probability. However, same as the model itself, CP is vulnerable to adversarial test examples (evasion) and perturbed calibration data (poisoning). We derive provably robust sets by bounding the worst-case change in conformity scores. Our tighter bounds lead to more efficient sets. We cover both continuous and discrete (sparse) data and our guarantees work both for evasion and poisoning attacks (on both features and labels).
Paper Structure (29 sections, 6 theorems, 48 equations, 9 figures, 4 tables, 1 algorithm)

This paper contains 29 sections, 6 theorems, 48 equations, 9 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Robust Prediction Sets
  4. Robustness to Evasion Attacks
  5. Robustness to Feature Poisoning Attacks
  6. Robustness to Label Poisoning Attacks
  7. Randomized Smoothing Bounds
  8. CAS: CDF-Aware Sets
  9. Finite Sample Correction
  10. Experiments
  11. Related Work
  12. Conclusion
  13. More On Conformal Prediction
  14. Impelementation Details
  15. Faster Evasion-Robustness via Calibration-time Bound
  16. ...and 14 more sections

Key Result

Theorem 2.1

If ${\mathcal{D}}_{\mathrm{cal}}=\{({\bm{x}}_i, y_i)\}_{i=1}^{n}$, and $({\bm{x}}_{n+1}, y_{n+1})$ are exchangeable, for any continuous score function $s: {\mathcal{X}} \times {\mathcal{Y}} \mapsto {\mathbb{R}}$ capturing the agreement between ${\bm{x}}$, and $y$, and user-specified $\alpha \in (0, where $q_\alpha := \mathrm{Quant}\left(\alpha;\{s({\bm{x}}_i, y_i)\}_{i = 1}^n\right)$ is the $\alp

Figures (9)

  • Figure 1: Empirical coverage [left] and average set size [middle] of RSCP and CAS for clean and perturbed data. All sets are certified robust up to radius $r=0.125$. [Right] Empirical coverage for different certified radii (on clean data). All results are for CIFAR-10 with Gaussian smoothing ($\sigma=0.25$). CAS is less conservative since it is closer to the nominal $1-\alpha$, and has smaller sets.
  • Figure 2: Average set size of CAS and RSCP under evasion for (from left to right) CIFAR-10, ImageNet (with TPS), and Cora-ML.
  • Figure 3: [Left] Set size for $r=0.12$ with different scores. [Middle] Maximum set size-preserving radius (average over test points). Both results are on CIFAR-10 dataset and $\sigma=0.25$. [Right] The effect of smoothing parameter $\sigma$ on the set size across a range of radii for CIFAR-10 dataset with error correction for $10^4$ samples.
  • Figure 4: [Left] Lower bound $1-\beta$ on the robust coverage of vanilla CP (\ref{['thrm:worst-case-coverage']}). CAS certifies a larger lower bound. [Middle] Distribution of prediction set sizes using the slower test-time vs. the faster calibration-time evasion certificate. [Right] Set sizes for RSCP and CAS with account for finite sample error. All results are for CIFAR-10 with $\sigma=0.25$.
  • Figure 5: [Left] The coverage of vanilla CP under calibration set with poisoned features. [Middle] The result of robust CP on the same calibration set. [Right] The average set size of the CP robust to feature poisoning for a range of nominal coverages given the clean calibration data, and $r=0.12$. All results are for CIFAR-10 dataset with $\sigma=0.25$.
  • ...and 4 more figures

Theorems & Definitions (12)

  • Theorem 2.1: Vovk2005AlgorithmicLI
  • Definition 3.1: Robust coverage
  • Proposition 3.1
  • Proposition 3.2
  • Proposition 5.1
  • Proposition 6.1
  • Proposition 6.2
  • proof : Proof of \ref{['thrm:conservative-guarantee']}
  • proof : Proof of \ref{['thrm:poisoning']}
  • proof : Proof of \ref{['thrm:worst-case-coverage']}
  • ...and 2 more