Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents

TL;DR

This paper addresses how to quantify and act on prediction uncertainty in risk-averse settings by linking conformal prediction sets to optimal actions. It establishes that prediction sets are a sufficient statistic for safe decision making under a value-at-risk objective, proves minimax optimality of a simple max-min action rule, and characterizes the optimal prediction sets via a population-level and a distribution-free finite-sample theory. The authors introduce Risk-Averse Calibration (RAC), a practical algorithm that derives action policies from predictions with distribution-free safety guarantees and demonstrate its advantages in medical diagnosis and recommender systems. The work advances a principled integration of uncertainty quantification and decision making for high-stakes applications, with potential extensions to conditional safety and robustness to mis-specified utilities.

Abstract

A fundamental question in data-driven decision making is how to quantify the uncertainty of predictions in ways that can usefully inform downstream action. This interface between prediction uncertainty and decision-making is especially important in risk-sensitive domains, such as medicine. In this paper, we develop decision-theoretic foundations that connect uncertainty quantification using prediction sets with risk-averse decision-making. Specifically, we answer three fundamental questions: (1) What is the correct notion of uncertainty quantification for risk-averse decision makers? We prove that prediction sets are optimal for decision makers who wish to optimize their value at risk. (2) What is the optimal policy that a risk averse decision maker should use to map prediction sets to actions? We show that a simple max-min decision policy is optimal for risk-averse decision makers. Finally, (3) How can we derive prediction sets that are optimal for such decision makers? We provide an exact characterization in the population regime and a distribution free finite-sample construction. Answering these questions naturally leads to an algorithm, Risk-Averse Calibration (RAC), which follows a provably optimal design for deriving action policies from predictions. RAC is designed to be both practical-capable of leveraging the quality of predictions in a black-box manner to enhance downstream utility-and safe-adhering to a user-defined risk threshold and optimizing the corresponding risk quantile of the user's downstream utility. Finally, we experimentally demonstrate the significant advantages of RAC in applications such as medical diagnosis and recommendation systems. Specifically, we show that RAC achieves a substantially improved trade-off between safety and utility, offering higher utility compared to existing methods while maintaining the safety guarantee.

Figures (3)

• Figure 1: RAC pipeline, an interface between prediction and action for high stakes applications.
• Figure 2: Left: Illustration of how the functions ${\boldsymbol \theta}$ and ${\boldsymbol a}$ are computed for a given $x \in \mathcal{X}$ and $t \in [0,1]$. Here, we have three actions $\mathcal{A} = \{a_1, a_2, a_3\}$ and four labels $\mathcal{Y} = \{y_1, y_2, y_3, y_4\}$. We also let $P_i := p(y_i|x)$ denote the conditional probabilities. For each of the actions $a_j$, $j=1,2,3$, the value $u^*_{a_j}$ is the $(1-t)$-quantile of the random variable $u(a_j, Y)$. The value ${\boldsymbol \theta}(x,t)$ corresponds to the maximum of these quantiles among the actions, and ${\boldsymbol a}(x,t)$ corresponds to the maximizing action. Right: Illustration of how the function ${\boldsymbol g}(x, \beta)$ is obtained from ${\boldsymbol \theta}(x,t)$ for a given $x$.
• Figure 3: Results from two experiments. (a) Average realized max-min value as a function of $\alpha$. (b) Fraction of wrong critical decisions: in medical diagnosis, severe omission of appropriate care (e.g., failing to act on pneumonia or COVID-19 cases); in MovieLens, the percentage of movies rated 1 or 2 that were incorrectly recommended. (c) Average realized utility. (d) Realized miscoverage.

Theorems & Definitions (10)

• Remark 2.1
• Proposition 2.2
• Theorem 2.3
• Proposition 3.1
• Theorem 3.2
• Theorem 4.1
• Corollary 4.2
• proof : Proof of Proposition \ref{['optimal_prediction_set']}
• Lemma A.1
• proof