Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximate full conformal prediction in RKHS

Davidson Lova Razafindrakoto, Alain Celisse, Jérôme Lacaille

TL;DR

This work tackles the computational intractability of full conformal prediction in RKHS by proposing a generic, non-asymptotic approximation framework that preserves distribution-free coverage. It builds a layered approach: starting from non-smooth losses, then exploiting smooth losses via local stability, and finally leveraging very smooth losses with influence functions to achieve tighter prediction regions. The authors introduce a quantifiable thickness measure to bound how far the approximate region can be from the full conformal region, and demonstrate that the approximations maintain coverage while shrinking the region size under increasing sample size and regularization. They provide concrete schemes for predictor approximation, score approximation, and region construction, along with explicit bounds and data-driven parameter-tuning strategies, and illustrate with robust loss examples like Logcosh, pseudo-Huber, and smoothed-pinball. The practical impact is a computable, provably reliable conformal prediction method in RKHS settings that adapts to loss smoothness to yield more informative confidence regions in high-dimensional, kernel-based learning tasks.

Abstract

Full conformal prediction is a framework that implicitly formulates distribution-free confidence prediction regions for a wide range of estimators. However, a classical limitation of the full conformal framework is the computation of the confidence prediction regions, which is usually impossible since it requires training infinitely many estimators (for real-valued prediction for instance). The main purpose of the present work is to describe a generic strategy for designing a tight approximation to the full conformal prediction region that can be efficiently computed. Along with this approximate confidence region, a theoretical quantification of the tightness of this approximation is developed, depending on the smoothness assumptions on the loss and score functions. The new notion of thickness is introduced for quantifying the discrepancy between the approximate confidence region and the full conformal one.

Approximate full conformal prediction in RKHS

TL;DR

This work tackles the computational intractability of full conformal prediction in RKHS by proposing a generic, non-asymptotic approximation framework that preserves distribution-free coverage. It builds a layered approach: starting from non-smooth losses, then exploiting smooth losses via local stability, and finally leveraging very smooth losses with influence functions to achieve tighter prediction regions. The authors introduce a quantifiable thickness measure to bound how far the approximate region can be from the full conformal region, and demonstrate that the approximations maintain coverage while shrinking the region size under increasing sample size and regularization. They provide concrete schemes for predictor approximation, score approximation, and region construction, along with explicit bounds and data-driven parameter-tuning strategies, and illustrate with robust loss examples like Logcosh, pseudo-Huber, and smoothed-pinball. The practical impact is a computable, provably reliable conformal prediction method in RKHS settings that adapts to loss smoothness to yield more informative confidence regions in high-dimensional, kernel-based learning tasks.

Abstract

Full conformal prediction is a framework that implicitly formulates distribution-free confidence prediction regions for a wide range of estimators. However, a classical limitation of the full conformal framework is the computation of the confidence prediction regions, which is usually impossible since it requires training infinitely many estimators (for real-valued prediction for instance). The main purpose of the present work is to describe a generic strategy for designing a tight approximation to the full conformal prediction region that can be efficiently computed. Along with this approximate confidence region, a theoretical quantification of the tightness of this approximation is developed, depending on the smoothness assumptions on the loss and score functions. The new notion of thickness is introduced for quantifying the discrepancy between the approximate confidence region and the full conformal one.
Paper Structure (49 sections, 46 theorems, 295 equations, 9 figures)

This paper contains 49 sections, 46 theorems, 295 equations, 9 figures.

Table of Contents

  1. State of the art
  2. Conformal prediction
  3. Full conformal prediction
  4. Approximations
  5. Split conformal papadopoulos2008inductive
  6. Cross-conformal vovk2015cross
  7. Designing a new computable approximation
  8. Predictor and loss functions
  9. Approximate full conformal prediction region
  10. First application to non-smooth losses
  11. Designing an approximation to the predictor
  12. Performance quantification
  13. Choice of non-conformity function
  14. Illustration
  15. Designing an approximation from a smooth loss
  16. ...and 34 more sections

Key Result

Theorem 2

vovk2022 If $(X_{1}, Y_{1}), \ldots, (X_{n+1}, Y_{n+1})$ are exchangeable and if $\hat{f}_D$ is invariant to permutations of the data in $D$, then the full conformal prediction region $\hat{C}_{\alpha}^{\mathrm{full}}(X_{n+1})$ enjoys the following coverage guarantee As such $\hat{C}_{\alpha}^{\mathrm{full}}(X_{n+1})$ is called a confidence prediction region. Moreover, if the non-conformity score

Figures (9)

  • Figure 1: Evolution of the upper bound in Eq. \ref{['eq.bound.thickness.non.smooth']} (dashed red line) and the quantity $\Delta^{(0)}$ (solid blue line) as a function of the sample size $n$ in $\log\log$ scale (to appreciate the rate). The data is sampled from $\mathrm{sklearn}$ synthetic dataset make_friedman1(sample_size=n). The kernel $k_\mathcal{H} \left( \cdot, \cdot \right)$ is set to be the Laplacian kernel (gamma=None). The loss function $\ell\left( \cdot, \cdot \right)$ is set to be the Logcosh Loss ($a=1.0$). The regularization parameter is set to decay as $\lambda \propto (n+1)^{-0.33}$. The fixed output value is set at $z=0$. The non-conformity function is set to be $s(\cdot, \cdot) : (y, u) \mapsto \left\lvert y - u \right\rvert$.
  • Figure 2: Evolution of the upper bound in Eq. \ref{['eq.bound.thickness.non.smooth']} (dashed red line) and the quantity $\Delta^{(1)}$ (solid blue line) as a function of the sample size $n$ in $\log\log$ scale (to appreciate the rate). The data is sampled from $\mathrm{sklearn}$ synthetic dataset make_friedman1(sample_size=n). The kernel $k_\mathcal{H} \left( \cdot, \cdot \right)$ is set to be the Laplacian kernel (gamma=None). The loss function $\ell\left( \cdot, \cdot \right)$ is set to be the Logcosh Loss ($a=1.0$). The regularization parameter is set to decay as $\lambda \propto (n+1)^{-0.33}$. The fixed output value is set at $z=0$. The non-conformity function is set to be $s(\cdot, \cdot) : (y, u) \mapsto \left\lvert y - u \right\rvert$.
  • Figure 3: Evolution of the two upper bounds on the thickness $\tilde{C}_{\lambda; \alpha}^{\mathrm{up} ,(2)}(X_{n+1})$ as a function of the sample size $n$ (in $\log\log$ scale to see the rate). The data is sampled from $\mathrm{sklearn}$ synthetic dataset make_friedman1(sample_size=n). The kernel $k_\mathcal{H} \left( \cdot, \cdot \right)$ is set to be the Laplacian kernel (gamma=None). The loss function $\ell\left( \cdot, \cdot \right)$ is set to be the Logcosh Loss ($a=1.0$). The regularization parameter is set to decay as $\lambda \propto (n+1)^{-0.33}$. The fixed output value is set at $z=0$. The non-conformity function is set to be $s(\cdot, \cdot) : (y, u) \mapsto \left\lvert y - u \right\rvert$.
  • Figure 4: Evolution of the upper bound in Eq. \ref{['eq.bound.thickness.non.smooth']} (dashed red line) and the quantity $\Delta^{(0)}$ (solid blue line) as a function of the sample size $n$ in $\log\log$ scale (to appreciate the rate). The data is sampled from $\mathrm{sklearn}$ synthetic dataset make_friedman1(sample_size=n). The kernel $k_\mathcal{H} \left( \cdot, \cdot \right)$ is set to be the Laplacian kernel (gamma=None). The loss function $\ell\left( \cdot, \cdot \right)$ is set to be the pseudo-Huber loss ($a=1.0$). The regularization parameter is set to decay as $\lambda \propto (n+1)^{-0.33}$. The fixed output value is set at $z=0$. The non-conformity function is set to be $s(\cdot, \cdot) : (y, u) \mapsto \left\lvert y - u \right\rvert$.
  • Figure 5: Evolution of the upper bound in Eq. \ref{['eq.bound.thickness.non.smooth']} (dashed red line) and the quantity $\Delta^{(1)}$ (solid blue line) as a function of the sample size $n$ in $\log\log$ scale (to appreciate the rate). The data is sampled from $\mathrm{sklearn}$ synthetic dataset make_friedman1(sample_size=n). The kernel $k_\mathcal{H} \left( \cdot, \cdot \right)$ is set to be the Laplacian kernel (gamma=None). The loss function $\ell\left( \cdot, \cdot \right)$ is set to be the pseudo-Huber loss ($a=1.0$). The regularization parameter is set to decay as $\lambda \propto (n+1)^{-0.33}$. The fixed output value is set at $z=0$. The non-conformity function is set to be $s(\cdot, \cdot) : (y, u) \mapsto \left\lvert y - u \right\rvert$.
  • ...and 4 more figures

Theorems & Definitions (57)

  • Definition 1
  • Theorem 2
  • Definition 3
  • Definition 4
  • Lemma 5
  • Definition 6
  • Lemma 7: Sandwiching
  • Theorem 11
  • Theorem 12
  • Corollary 13
  • ...and 47 more