Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Transport-based Conformal Prediction

Gauthier Thurin, Kimia Nadjahi, Claire Boyer

TL;DR

This work introduces a novel CP procedure handling multivariate score functions through the lens of optimal transport, and proves that its approach ensures finite-sample, distribution-free coverage properties, similar to typical CP methods.

Abstract

Conformal Prediction (CP) is a principled framework for quantifying uncertainty in blackbox learning models, by constructing prediction sets with finite-sample coverage guarantees. Traditional approaches rely on scalar nonconformity scores, which fail to fully exploit the geometric structure of multivariate outputs, such as in multi-output regression or multiclass classification. Recent methods addressing this limitation impose predefined convex shapes for the prediction sets, potentially misaligning with the intrinsic data geometry. We introduce a novel CP procedure handling multivariate score functions through the lens of optimal transport. Specifically, we leverage Monge-Kantorovich vector ranks and quantiles to construct prediction region with flexible, potentially non-convex shapes, better suited to the complex uncertainty patterns encountered in multivariate learning tasks. We prove that our approach ensures finite-sample, distribution-free coverage properties, similar to typical CP methods. We then adapt our method for multi-output regression and multiclass classification, and also propose simple adjustments to generate adaptive prediction regions with asymptotic conditional coverage guarantees. Finally, we evaluate our method on practical regression and classification problems, illustrating its advantages in terms of (conditional) coverage and efficiency.

Optimal Transport-based Conformal Prediction

TL;DR

This work introduces a novel CP procedure handling multivariate score functions through the lens of optimal transport, and proves that its approach ensures finite-sample, distribution-free coverage properties, similar to typical CP methods.

Abstract

Conformal Prediction (CP) is a principled framework for quantifying uncertainty in blackbox learning models, by constructing prediction sets with finite-sample coverage guarantees. Traditional approaches rely on scalar nonconformity scores, which fail to fully exploit the geometric structure of multivariate outputs, such as in multi-output regression or multiclass classification. Recent methods addressing this limitation impose predefined convex shapes for the prediction sets, potentially misaligning with the intrinsic data geometry. We introduce a novel CP procedure handling multivariate score functions through the lens of optimal transport. Specifically, we leverage Monge-Kantorovich vector ranks and quantiles to construct prediction region with flexible, potentially non-convex shapes, better suited to the complex uncertainty patterns encountered in multivariate learning tasks. We prove that our approach ensures finite-sample, distribution-free coverage properties, similar to typical CP methods. We then adapt our method for multi-output regression and multiclass classification, and also propose simple adjustments to generate adaptive prediction regions with asymptotic conditional coverage guarantees. Finally, we evaluate our method on practical regression and classification problems, illustrating its advantages in terms of (conditional) coverage and efficiency.

Paper Structure

This paper contains 36 sections, 4 theorems, 38 equations, 16 figures, 1 table.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Methodology
  4. Setting
  5. Optimal transport toolbox
  6. OT-based conformal prediction (OT-CP)
  7. Coverage guarantees
  8. Multi-Output Regression
  9. OT-CP can output non-convex prediction sets
  10. CP methods for multi-output regression.
  11. OT-CP for multi-output regression.
  12. Numerical experiments.
  13. OT-CP+: an adaptive version
  14. Methodology.
  15. Experiments on simulated data.
  16. ...and 21 more sections

Key Result

Theorem 2.4

Suppose $\{(X_i, Y_i)\}_{i=1}^n \cup \{(X_{\rm test}, Y_{\rm test})\}$ are exchangeable. Let $\alpha \in (0,1)$ such that $\lceil \alpha(n_2+1) \rceil \leq n_2$. The prediction region $\widehat{\mathcal{C}}_\alpha$ constructed on $\{(X_i, Y_i)\}_{i=1}^n$ satisfies where the probability is taken over the joint distribution of $\{(X_i, Y_i)\}_{i=1}^n \cup \{(X_{\rm test}, Y_{\rm test})\}$ and $n_{\

Figures (16)

  • Figure 1: Ranking multivariate scores using optimal transport. The colormap encodes how the 2-dimensional scores $\{S_i\}_{i=1}^n$ in (a) are transported onto the reference rank vectors $\{U_i\}_{i=1}^n$ in (b).
  • Figure 2: Conformal multi-output regression with OT-CP on simulated data
  • Figure 3: Adaptive conformal regression with OT-CP+
  • Figure 4: Conditional coverage on real datasets of two adaptive conformal procedures for multi-output regression
  • Figure 5: Ordering must depend on the chosen scores: (a) Center-outward for signed errors, (b) Left-to-right for absolute errors
  • ...and 11 more figures

Theorems & Definitions (14)

  • Example 1: Multi-output regression
  • Example 2: Multiclass classification
  • Definition 2.1: Empirical Monge-Kantorovich ranks, chernozhukov2015mongekantorovichHallinAOS_2021
  • Remark 2.2
  • Remark 2.3: Computational aspects
  • Theorem 2.4: Coverage guarantee
  • Theorem 3.2
  • proof : Proof of \ref{['thmMarginalCoverage']}
  • Lemma B.1
  • proof
  • ...and 4 more