Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distribution-Free Predictive Inference under Unknown Temporal Drift

Elise Han, Chengpiao Huang, Kaizheng Wang

TL;DR

This work tackles distribution-free predictive inference under unknown temporal drift by reducing the problem to quantile estimation of conformity scores. It introduces an adaptive rolling window (ARW) that selects the look-back period via a data-driven bias-variance proxy inspired by Goldenshluger-Lepski, yielding quantile estimates with sharp drift-adaptive guarantees. Theoretical results provide an oracle-type bound and training-conditional coverage guarantees, while extensive synthetic and real-data experiments demonstrate robust performance under various drift patterns. The approach enables reliable, model-agnostic prediction sets in nonstationary environments with practical applicability to real-world forecasting tasks.

Abstract

Distribution-free prediction sets play a pivotal role in uncertainty quantification for complex statistical models. Their validity hinges on reliable calibration data, which may not be readily available as real-world environments often undergo unknown changes over time. In this paper, we propose a strategy for choosing an adaptive window and use the data therein to construct prediction sets. The window is selected by optimizing an estimated bias-variance tradeoff. We provide sharp coverage guarantees for our method, showing its adaptivity to the underlying temporal drift. We also illustrate its efficacy through numerical experiments on synthetic and real data.

Distribution-Free Predictive Inference under Unknown Temporal Drift

TL;DR

This work tackles distribution-free predictive inference under unknown temporal drift by reducing the problem to quantile estimation of conformity scores. It introduces an adaptive rolling window (ARW) that selects the look-back period via a data-driven bias-variance proxy inspired by Goldenshluger-Lepski, yielding quantile estimates with sharp drift-adaptive guarantees. Theoretical results provide an oracle-type bound and training-conditional coverage guarantees, while extensive synthetic and real-data experiments demonstrate robust performance under various drift patterns. The approach enables reliable, model-agnostic prediction sets in nonstationary environments with practical applicability to real-world forecasting tasks.

Abstract

Distribution-free prediction sets play a pivotal role in uncertainty quantification for complex statistical models. Their validity hinges on reliable calibration data, which may not be readily available as real-world environments often undergo unknown changes over time. In this paper, we propose a strategy for choosing an adaptive window and use the data therein to construct prediction sets. The window is selected by optimizing an estimated bias-variance tradeoff. We provide sharp coverage guarantees for our method, showing its adaptivity to the underlying temporal drift. We also illustrate its efficacy through numerical experiments on synthetic and real data.
Paper Structure (28 sections, 8 theorems, 74 equations, 4 figures, 6 tables, 4 algorithms)

This paper contains 28 sections, 8 theorems, 74 equations, 4 figures, 6 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Main contributions.
  3. Related works.
  4. Notation.
  5. Outline.
  6. Problem setup
  7. Predictive inference
  8. Reduction to quantile estimation under temporal drift
  9. Methodology
  10. A bias-variance decomposition for the sample quantile
  11. Adaptive rolling window for quantile estimation and predictive inference
  12. Theoretical analysis
  13. Numerical experiments
  14. Synthetic data
  15. Gaussian mean estimation.
  16. ...and 13 more sections

Key Result

Theorem 3.1

Let Assumption assumption-AC hold. Fix $k\in[t]$. Choose $\delta\in(0,1)$. Define With probability at least $1-\delta$,

Figures (4)

  • Figure 1: True means $\mu_t$.
  • Figure 2: Mean absolute errors of coverage for mean estimation. Left: stationary case. Right: non-stationary case.
  • Figure 3: Mean absolute errors of coverage for linear regression. Left: stationary case. Right: non-stationary case.
  • Figure 4: Weekly average of logarithmic prices.

Theorems & Definitions (17)

  • Definition 3.1: Left and right quantiles
  • Theorem 3.1: Bias-variance decomposition
  • Theorem 3.2: Empirical bias-variance decomposition
  • Theorem 4.1: Oracle inequality for \ref{['alg-quantile']}
  • Example 4.1: Change point
  • Example 4.2: Bounded drift
  • proof : Proof sketch for \ref{['thm-GL']}
  • Corollary 4.1
  • Lemma B.1: Perturbation bound for empirical quantile
  • proof : Proof of \ref{['lem-max-diff-order-stats']}
  • ...and 7 more