Conformal prediction with local weights: randomization enables local guarantees

TL;DR

The paper tackles distribution-free predictive intervals with finite-sample conditional guarantees and introduces randomly localized conformal prediction (RLCP) to achieve approximate test-conditional coverage and robustness to covariate shift while preserving marginal validity. RLCP randomizes the localization center, enabling a theoretical framework that yields marginal coverage guarantees and provable bounds on local coverage and covariate-shift performance. The authors compare RLCP with baseLCP and calLCP through extensive simulations and a real abalones dataset, demonstrating better local adaptivity and more stable conditional coverage, especially at small bandwidths. Overall, RLCP provides a principled, randomization-based pathway to balance local coverage, covariate shift resilience, and practical interval width in distribution-free prediction.

Abstract

In this work, we consider the problem of building distribution-free prediction intervals with finite-sample conditional coverage guarantees. Conformal prediction (CP) is an increasingly popular framework for building such intervals with distribution-free guarantees, but these guarantees only ensure marginal coverage: the probability of coverage is averaged over both the training and test data, meaning that there might be substantial undercoverage within certain subpopulations. Instead, ideally we would want to have local coverage guarantees that hold for each possible value of the test point's features. While the impossibility of achieving pointwise local coverage is well established in the literature, many variants of conformal prediction algorithm show favourable local coverage properties empirically. Relaxing the definition of local coverage can allow for a theoretical understanding of this empirical phenomenon. We propose randomly localized conformal prediction (RLCP), a method that builds on localized CP and weighted CP techniques to return prediction intervals that are not only marginally valid but also offer relaxed local coverage guarantees and validity under covariate shift. Through a series of simulations and real data experiments, we validate these coverage guarantees of RLCP while comparing it with the other local conformal prediction methods.

Figures (13)

• Figure 1: Marginal coverage of each method in two univariate simulation settings, across different bandwidths. Results are shown for $50$ independent trials. See Section \ref{['sec:simulation_univariate']} for details.
• Figure 2: Top panels: the average prediction intervals produced by each method (at each feature value $x$, the endpoints of the prediction interval are averaged over $50$ independent trials), compared to the oracle prediction interval. Bottom panels: The corresponding local coverage of each method, averaged over $50$ independent trials. See Section \ref{['sec:simulation_univariate']} for details.
• Figure 3: Predictive coverage conditional on $X_{n+1}\in B$, for three choices of the set $B$ (namely, $B_{\textnormal{in}}$, $B_{\textnormal{out}}$, and $B_{\textnormal{in}}\cup B_{\textnormal{out}} = {\mathcal{X}}$). Results for each method with fixed bandwidth $h=1.5$ are plotted with respect to dimension $d$ of the feature space. Results are averaged over $50$ independent trials. See Section \ref{['sec:simulation_multivariate']} for details.
• Figure 4: Predictive coverage conditional on $X_{n+1}\in B$, for three choices of the set $B$ (namely, $B_{\textnormal{in}}$, $B_{\textnormal{out}}$, and $B_{\textnormal{in}}\cup B_{\textnormal{out}} = {\mathcal{X}}$). Results for each method, averaged over $50$ independent trials, are plotted with respect to dimension $d$ of the feature space. Each method is run with its own bandwidth choice, so that a constant $50$ effective sample size is maintained across dimensions. See Section \ref{['sec:simulation_multivariate']} for more experimental details.
• Figure 5: Marginal coverage (averaged over the test set) and prediction interval width (median over the test set), for each of the three methods across different bandwidths and for different choices of the pretrained base predictor $\widehat{f}$, for the real data experiment. Results are averaged over $50$ random splits of the data into pretraining, calibration, and test sets. See Section \ref{['sec:real_data']} for details.

Theorems & Definitions (20)

• Proposition 1: Key property of RLCP
• Theorem 1: Marginal coverage guarantee
• proof : Proof of Theorem \ref{['thm:marginal_coverage']}
• proof : Proof of Proposition \ref{['prop:key_property']}
• Theorem 2: Test-conditional coverage guarantee
• Corollary 1
• Theorem 3: Robustness to covariate shift
• Proposition 2
• Theorem A.4
• proof : Proof of Theorem \ref{['thm:training_conditional']}