Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distribution-informed Online Conformal Prediction

Dongjian Hu, Junxi Wu, Shu-Tao Xia, Changliang Zou

TL;DR

The authors address uncertainty quantification under distribution shifts in online conformal prediction by introducing Conformal Optimistic Prediction (COP), which refines online thresholds using an estimated CDF of non-conformity scores. By linking COP to Optimistic Online Gradient Descent, they derive joint regret–coverage bounds and prove distribution-free coverage with asymptotic consistency under i.i.d. scores. Empirical results on simulated drift, financial, energy, and climate data show COP maintains target coverage while delivering tighter prediction intervals than existing baselines. The approach combines model-agnostic CP guarantees with data-pattern awareness to improve efficiency in streaming settings, making it practical for real-time uncertainty quantification.

Abstract

Conformal prediction provides a pivotal and flexible technique for uncertainty quantification by constructing prediction sets with a predefined coverage rate. Many online conformal prediction methods have been developed to address data distribution shifts in fully adversarial environments, resulting in overly conservative prediction sets. We propose Conformal Optimistic Prediction (COP), an online conformal prediction algorithm incorporating underlying data pattern into the update rule. Through estimated cumulative distribution function of non-conformity scores, COP produces tighter prediction sets when predictable pattern exists, while retaining valid coverage guarantees even when estimates are inaccurate. We establish a joint bound on coverage and regret, which further confirms the validity of our approach. We also prove that COP achieves distribution-free, finite-sample coverage under arbitrary learning rates and can converge when scores are $i.i.d.$. The experimental results also show that COP can achieve valid coverage and construct shorter prediction intervals than other baselines.

Distribution-informed Online Conformal Prediction

TL;DR

The authors address uncertainty quantification under distribution shifts in online conformal prediction by introducing Conformal Optimistic Prediction (COP), which refines online thresholds using an estimated CDF of non-conformity scores. By linking COP to Optimistic Online Gradient Descent, they derive joint regret–coverage bounds and prove distribution-free coverage with asymptotic consistency under i.i.d. scores. Empirical results on simulated drift, financial, energy, and climate data show COP maintains target coverage while delivering tighter prediction intervals than existing baselines. The approach combines model-agnostic CP guarantees with data-pattern awareness to improve efficiency in streaming settings, making it practical for real-time uncertainty quantification.

Abstract

Conformal prediction provides a pivotal and flexible technique for uncertainty quantification by constructing prediction sets with a predefined coverage rate. Many online conformal prediction methods have been developed to address data distribution shifts in fully adversarial environments, resulting in overly conservative prediction sets. We propose Conformal Optimistic Prediction (COP), an online conformal prediction algorithm incorporating underlying data pattern into the update rule. Through estimated cumulative distribution function of non-conformity scores, COP produces tighter prediction sets when predictable pattern exists, while retaining valid coverage guarantees even when estimates are inaccurate. We establish a joint bound on coverage and regret, which further confirms the validity of our approach. We also prove that COP achieves distribution-free, finite-sample coverage under arbitrary learning rates and can converge when scores are . The experimental results also show that COP can achieve valid coverage and construct shorter prediction intervals than other baselines.

Paper Structure

This paper contains 32 sections, 7 theorems, 78 equations, 2 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Problem Setup
  4. Conformal Prediction for Sequential Data
  5. Optimistic Online Gradient Descent
  6. Method and Theory
  7. Conformal Optimistic Prediction
  8. Connection with OOGD
  9. Coverage Guarantees
  10. Experiments
  11. Setup
  12. Simulation Dataset
  13. Real-world Datasets
  14. Conclusions
  15. Some Discussions
  16. ...and 17 more sections

Key Result

Proposition 1

Assume that $\hat{F}_{t+1}(\hat{q}_{t+1})-(1-\alpha)$ and $F_{t+1}(\hat{q}_{t+1})-(1-\alpha)$ have the same sign, and $F_{t+1}$ is $L$-Lipschitz continuous. With a suitably small $\lambda_{t+1}>0$, we have:

Figures (2)

  • Figure 1: Comparison results of coverage rate on Amazon stock dataset with Prophet model. The coverage is averaged over a rolling window of 50 points.
  • Figure 2: Comparison results of prediction sets on Amazon stock dataset with Prophet model.

Theorems & Definitions (13)

  • Proposition 1
  • Theorem 1
  • Proposition 2
  • Theorem 2
  • Theorem 3
  • proof
  • proof
  • Lemma 1
  • proof
  • Lemma 2
  • ...and 3 more