Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Uncertainty Sets: Leveraging Optimal Transport to Extend Conformal Predictive Distribution to Multivariate Settings

Eugene Ndiaye

TL;DR

This work tackles extending conformal prediction (CP) from scalar to multivariate outcomes by leveraging optimal transport (OT) to define center-outward ranks and multivariate quantile regions. It introduces a conformalized vector-valued score framework where the candidate’s rank is defined via an augmented OT map, ensuring exact finite-sample, distribution-free coverage. A key technical contribution is a tractable algorithm that precomputes a fixed polyhedral partition of the score space, enabling efficient prediction-set construction without solving OT for every candidate. Building on this backbone, the authors construct multivariate conformal predictive distributions (CPDs) with finite-sample calibration, including randomized variants that generalize the Dempster–Hill procedure to higher dimensions. Collectively, the paper provides distribution-free multivariate predictive distributions and calibrated prediction regions, advancing principled uncertainty quantification for multivariate tasks with practical, scalable algorithms.

Abstract

Conformal prediction (CP) constructs uncertainty sets for model outputs with finite-sample coverage guarantees. A candidate output is included in the prediction set if its non-conformity score is not considered extreme relative to the scores observed on a set of calibration examples. However, this procedure is only straightforward when scores are scalar-valued, which has limited CP to real-valued scores or ad-hoc reductions to one dimension. The problem of ordering vectors has been studied via optimal transport (OT), which provides a principled method for defining vector-ranks and multivariate quantile regions, though typically with only asymptotic coverage guarantees. We restore finite-sample, distribution-free coverage by conformalizing the vector-valued OT quantile region. Here, a candidate's rank is defined via a transport map computed for the calibration scores augmented with that candidate's score. This defines a continuum of OT problems for which we prove that the resulting optimal assignment is piecewise-constant across a fixed polyhedral partition of the score space. This allows us to characterize the entire prediction set tractably, and provides the machinery to address a deeper limitation of prediction sets: that they only indicate which outcomes are plausible, but not their relative likelihood. In one dimension, conformal predictive distributions (CPDs) fill this gap by producing a predictive distribution with finite-sample calibration. Extending CPDs beyond one dimension remained an open problem. We construct, to our knowledge, the first multivariate CPDs with finite-sample calibration, i.e., they define a valid multivariate distribution where any derived uncertainty region automatically has guaranteed coverage. We present both conservative and exact randomized versions, the latter resulting in a multivariate generalization of the classical Dempster-Hill procedure.

Beyond Uncertainty Sets: Leveraging Optimal Transport to Extend Conformal Predictive Distribution to Multivariate Settings

TL;DR

This work tackles extending conformal prediction (CP) from scalar to multivariate outcomes by leveraging optimal transport (OT) to define center-outward ranks and multivariate quantile regions. It introduces a conformalized vector-valued score framework where the candidate’s rank is defined via an augmented OT map, ensuring exact finite-sample, distribution-free coverage. A key technical contribution is a tractable algorithm that precomputes a fixed polyhedral partition of the score space, enabling efficient prediction-set construction without solving OT for every candidate. Building on this backbone, the authors construct multivariate conformal predictive distributions (CPDs) with finite-sample calibration, including randomized variants that generalize the Dempster–Hill procedure to higher dimensions. Collectively, the paper provides distribution-free multivariate predictive distributions and calibrated prediction regions, advancing principled uncertainty quantification for multivariate tasks with practical, scalable algorithms.

Abstract

Conformal prediction (CP) constructs uncertainty sets for model outputs with finite-sample coverage guarantees. A candidate output is included in the prediction set if its non-conformity score is not considered extreme relative to the scores observed on a set of calibration examples. However, this procedure is only straightforward when scores are scalar-valued, which has limited CP to real-valued scores or ad-hoc reductions to one dimension. The problem of ordering vectors has been studied via optimal transport (OT), which provides a principled method for defining vector-ranks and multivariate quantile regions, though typically with only asymptotic coverage guarantees. We restore finite-sample, distribution-free coverage by conformalizing the vector-valued OT quantile region. Here, a candidate's rank is defined via a transport map computed for the calibration scores augmented with that candidate's score. This defines a continuum of OT problems for which we prove that the resulting optimal assignment is piecewise-constant across a fixed polyhedral partition of the score space. This allows us to characterize the entire prediction set tractably, and provides the machinery to address a deeper limitation of prediction sets: that they only indicate which outcomes are plausible, but not their relative likelihood. In one dimension, conformal predictive distributions (CPDs) fill this gap by producing a predictive distribution with finite-sample calibration. Extending CPDs beyond one dimension remained an open problem. We construct, to our knowledge, the first multivariate CPDs with finite-sample calibration, i.e., they define a valid multivariate distribution where any derived uncertainty region automatically has guaranteed coverage. We present both conservative and exact randomized versions, the latter resulting in a multivariate generalization of the classical Dempster-Hill procedure.

Paper Structure

This paper contains 47 sections, 15 theorems, 172 equations, 3 figures.

Table of Contents

  1. Introduction
  2. The Challenge of Vector-Valued Scores
  3. Beyond Prediction Sets
  4. Calibrated Predictive Distributions
  5. Conformal Predictive Distributions (CPDs)
  6. Contributions Overview
  7. Related Work
  8. Scalarizing vector scores with norms.
  9. Optimal-transport merging with a pre-fit transform.
  10. Copula-based multivariate modeling.
  11. Prediction Sets from the Ground-Truth Distribution
  12. Prediction Sets from the Empirical Distribution
  13. Using Uniform Convergence of Empirical Distribution
  14. The Conformal Approach
  15. Predictive Distribution Based on Confidence Distribution
  16. ...and 32 more sections

Key Result

Lemma 2.1

If $\mathcal{Z}_{n+1} = (Z_1, \dots, Z_n, Z)$ are real-valued exchangeable random variables with no ties, then $F_n(Z)$ follow a discrete uniform measure on a regular grid of $n+1$ points on $[0,1]$ i.e. That is to say, denoting $\mathbb{P}^{(n+1)}$ as the joint law of $\mathcal{Z}_{n+1}$, it holds

Figures (3)

  • Figure 1: Illustration of the active cells $\Omega_r$ for various radius. It is bounded whenever $r<1$ and might be unbounded otherwise. In uncertainty quantification, $r=r_\alpha$ is the radius corresponding to a level of confidence $1-\alpha$ and is strictly smaller than one for every $\alpha \in [0, 1).$
  • Figure 2: Anisotropic Gaussian Distribution. Illustratation of the Multivariate conformalized quantile region when the conformity scores $\mathbf{Z}$ are generated from a mean-centered Gaussian distribution. The prediction model is the null model $\hat{\mathbf{y}}(\mathbf{x}) = \mathbf{0}$ and the covariance structure is set as $\mathbf{Y} \sim \mathcal{N}(\mathbf{0}, \Sigma)$, where $\Sigma = 10.80.81$.
  • Figure 3: A skewed distribution. We illustrate geometric adaptivity when the scores $\mathbf{Z} = (Z_1, Z_2)^\top$ are generated from a nonlinear transformation of standard normal variables. The scores follow $\mathbf{Y} = R (X_{\text{err}},\, Y_{\text{err}})^\top$, where $Y_{\text{err}} = 15 Z_2 + 24(Z_1^2 - 1)$ and $R$ is a $45^\circ$ rotation matrix.

Theorems & Definitions (33)

  • Lemma 2.1
  • Proposition 2.1
  • Definition 2.1: Predictive Distribution Function shen2018prediction
  • Definition 3.1: Multivariate Distribution and Rank hallin2021
  • Proposition 4.1: Coverage of the Conformalized Quantile Region
  • Proposition 4.2
  • Proposition 4.3
  • Proposition 4.4: Polyhedral Partition
  • Proposition 4.5: Boundness of the Conformal Quantile Region
  • Definition 4.1: Multivariate Predictive Distribution Map
  • ...and 23 more