Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Conformal Inference with Retrospective Adjustment for Faster Adaptation to Distribution Shift

Jungbin Jun, Ilsang Ohn

TL;DR

This work tackles online uncertainty quantification under distribution shift by integrating retrospective adjustment into conformal inference. By employing a time-windowed kernel ridge regression base and Jackknife+ with efficient leave-one-out updates, the method retroactively revises past residuals and predictions to align with the current data-generating process. Theoretical guarantees extend long-run coverage under adaptive-conformal regimes, while extensive simulations and real-data analyses show faster coverage calibration and tighter prediction intervals than forward-updating baselines. The approach offers a scalable path to robust, distribution-shift-resilient prediction sets in online settings with practical impact for streaming applications.

Abstract

Conformal prediction has emerged as a powerful framework for constructing distribution-free prediction sets with guaranteed coverage assuming only the exchangeability assumption. However, this assumption is often violated in online environments where data distributions evolve over time. Several recent approaches have been proposed to address this limitation, but, typically, they slowly adapt to distribution shifts because they update predictions only in a forward manner, that is, they generate a prediction for a newly observed data point while previously computed predictions are not updated. In this paper, we propose a novel online conformal inference method with retrospective adjustment, which is designed to achieve faster adaptation to distributional shifts. Our method leverages regression approaches with efficient leave-one-out update formulas to retroactively adjust past predictions when new data arrive, thereby aligning the entire set of predictions with the most recent data distribution. Through extensive numerical studies performed on both synthetic and real-world data sets, we show that the proposed approach achieves faster coverage recalibration and improved statistical efficiency compared to existing online conformal prediction methods.

Online Conformal Inference with Retrospective Adjustment for Faster Adaptation to Distribution Shift

TL;DR

This work tackles online uncertainty quantification under distribution shift by integrating retrospective adjustment into conformal inference. By employing a time-windowed kernel ridge regression base and Jackknife+ with efficient leave-one-out updates, the method retroactively revises past residuals and predictions to align with the current data-generating process. Theoretical guarantees extend long-run coverage under adaptive-conformal regimes, while extensive simulations and real-data analyses show faster coverage calibration and tighter prediction intervals than forward-updating baselines. The approach offers a scalable path to robust, distribution-shift-resilient prediction sets in online settings with practical impact for streaming applications.

Abstract

Conformal prediction has emerged as a powerful framework for constructing distribution-free prediction sets with guaranteed coverage assuming only the exchangeability assumption. However, this assumption is often violated in online environments where data distributions evolve over time. Several recent approaches have been proposed to address this limitation, but, typically, they slowly adapt to distribution shifts because they update predictions only in a forward manner, that is, they generate a prediction for a newly observed data point while previously computed predictions are not updated. In this paper, we propose a novel online conformal inference method with retrospective adjustment, which is designed to achieve faster adaptation to distributional shifts. Our method leverages regression approaches with efficient leave-one-out update formulas to retroactively adjust past predictions when new data arrive, thereby aligning the entire set of predictions with the most recent data distribution. Through extensive numerical studies performed on both synthetic and real-world data sets, we show that the proposed approach achieves faster coverage recalibration and improved statistical efficiency compared to existing online conformal prediction methods.

Paper Structure

This paper contains 39 sections, 9 theorems, 42 equations, 8 figures, 6 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Conformal prediction for exchangeable data
  4. Split Conformal Prediction
  5. Full Conformal Prediction
  6. Jackknife+ Conformal Prediction
  7. Adaptive Conformal Inference
  8. Original ACI
  9. Step Size Tuning Methods for ACI
  10. Regression with Linear Smoother
  11. Linear smoother
  12. Kernel ridge regression
  13. Leave-one-out formula
  14. Online Conformal Inference with Retrospective Adjustment
  15. Jackknife+ with Efficient Leave-One-Out Formula
  16. ...and 24 more sections

Key Result

Lemma 1

For a linear smoother with the self-stable property, the leave-one-out residual is given by for each $i \in [n]$.

Figures (8)

  • Figure 1: Coverage and prediction interval width of the proposed RetroAdj and forward online conformal inference methods (FW) over five ACI-based algorithms. Each bar represents the average over all time steps and simulation replications and each error bar denote the interquartile range.
  • Figure 2: Local coverage and prediction interval width of the proposed RetroAdj and forward online conformal inference methods (FW) for Setting 1 and 2. For all methods, the DtACI algorithm is employed to adjust the miscoverage level.
  • Figure 3: Coverage and prediction interval width of the proposed RetroAdj and forward online conformal inference methods (FW) for Setting 1.
  • Figure 4: Local coverage and prediction interval width of the proposed RetroAdj and forward online conformal inference methods (FW) for the prediction of per-capita violent crime rate. For all methods, the DtACI algorithm is employed to adjust the miscoverage level.
  • Figure 5: Local coverage and prediction interval width of the proposed RetroAdj and forward online conformal inference methods (FW) for the prediction of the electricity price. For all methods, the DtACI algorithm is employed to adjust the miscoverage level.
  • ...and 3 more figures

Theorems & Definitions (18)

  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • Lemma 4
  • proof
  • Theorem 5
  • proof
  • ...and 8 more