Optimal Conformal Prediction under Epistemic Uncertainty

TL;DR

This work addresses how to incorporate epistemic uncertainty into conformal prediction by leveraging second-order representations. It introduces Bernoulli Prediction Sets (BPS), a provably optimal, randomized set predictor that yields the smallest possible conditional-coverage sets under credal (second-order) distributions; when only a single first-order predictor is available, BPS recovers Adaptive Prediction Sets (APS). The authors further adapt BPS to scenarios where second-order validity may be compromised by applying conformal risk control to preserve marginal coverage, and provide an efficient linear-programming formulation that generalizes APS. Empirical results on synthetic data and CIFAR-10/100 with various second-order predictors show that BPS achieves superior or comparable conditional coverage with competitive set sizes, especially in regions of high epistemic uncertainty, illustrating the practical impact of optimally accounting for epistemic uncertainty in CP. The work advances uncertainty quantification for safety-critical applications by offering a principled, scalable method to algebraically blend second-order uncertainty with conformal guarantees.

Abstract

Conformal prediction (CP) is a popular frequentist framework for representing uncertainty by providing prediction sets that guarantee coverage of the true label with a user-adjustable probability. In most applications, CP operates on confidence scores coming from a standard (first-order) probabilistic predictor (e.g., softmax outputs). Second-order predictors, such as credal set predictors or Bayesian models, are also widely used for uncertainty quantification and are known for their ability to represent both aleatoric and epistemic uncertainty. Despite their popularity, there is still an open question on ``how they can be incorporated into CP''. In this paper, we discuss the desiderata for CP when valid second-order predictions are available. We then introduce Bernoulli prediction sets (BPS), which produce the smallest prediction sets that ensure conditional coverage in this setting. When given first-order predictions, BPS reduces to the well-known adaptive prediction sets (APS). Furthermore, when the validity assumption on the second-order predictions is compromised, we apply conformal risk control to obtain a marginal coverage guarantee while still accounting for epistemic uncertainty.

Figures (4)

• Figure 1: Comparison of BPS (ours) and APS romano2020classification in terms of conditional coverage, given a second-order prediction (i.e., a credal set with vertices shown as black circles). BPS uses the vertices as input, while APS, requiring a single distribution, is applied to the mean of the credal set (the blue square ${\bm{\pi}} = [0.5, 0.2, 0.3]$). The output of each method is shown as the probability of including each label in the set for a nominal coverage of $0.9$. For each method, the conditional coverage under any distribution on the simplex is shown below. Inside the given credal set, APS has both under-coverage (red) and over-coverage (green) areas. This is while for BPS the entire credal set is green (coverage $\ge 1 - \alpha$).
• Figure 2: Comparison of BPS vs. APS at nominal coverage $1-\alpha=0.9$ given valid credal sets. Here, BPS constructs prediction sets using the vertices of the credal sets, while APS uses the mean first-order distribution. The parameter $d$ denotes the radius of the credal sets. While APS conditional coverage is spread out around $1 - \alpha$, for BPS, the conditional coverage is always higher.
• Figure 3: Comparing coverage of BPS and APS across various levels of epistemic and aleatoric uncertainty; the results are on the CIFAR-10 dataset using the ensemble model with $1 - \alpha = 0.9$. Here, data points are binned based on their AU and EU into 100 equal-sized bins (empty bins are shown in gray). APS has more intense red bins. This means that there are more regions (characterized by AU and EU) where APS has lower coverage. Note that the color bar is unbalanced $[\le-0.3, 0]$ is red and $[0, 0.1]$ is green.
• Figure 4: Coverage and set size comparison of the baselines for the synthetic example of the APS paper.

Theorems & Definitions (9)

• Definition 1: valid second-order predictions
• Proposition 3.1
• Proposition 4.1
• Proposition 4.2
• proof
• proof
• proof
• Theorem B.1
• proof