Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Predictive inference for time series: why is split conformal effective despite temporal dependence?

Rina Foygel Barber, Ashwin Pananjady

TL;DR

The paper develops a theoretical framework to explain why split conformal prediction remains effective for time series with temporal dependence and memory-based predictors. It introduces the switch coefficient to quantify deviations from exchangeability caused by dependence and shows that coverage loss for pretrained and split conformal methods can be bounded by this coefficient, with sharp results for stationary $\\beta$-mixing processes. The authors derive matching lower bounds, establish conditions preventing undercovering or overcovering, and extend the results to memory-enabled score functions, demonstrating faster (linear) rates compared to prior $\\sqrt{\\tau/n}$-type bounds. Overall, the work provides a principled, quantitative explanation for the empirical success of split conformal in dependent data settings and offers tools potentially applicable to broader non-exchangeable predictive inference methods.

Abstract

We consider the problem of uncertainty quantification for prediction in a time series: if we use past data to forecast the next time point, can we provide valid prediction intervals around our forecasts? To avoid placing distributional assumptions on the data, in recent years the conformal prediction method has been a popular approach for predictive inference, since it provides distribution-free coverage for any iid or exchangeable data distribution. However, in the time series setting, the strong empirical performance of conformal prediction methods is not well understood, since even short-range temporal dependence is a strong violation of the exchangeability assumption. Using predictors with "memory" -- i.e., predictors that utilize past observations, such as autoregressive models -- further exacerbates this problem. In this work, we examine the theoretical properties of split conformal prediction in the time series setting, including the case where predictors may have memory. Our results bound the loss of coverage of these methods in terms of a new "switch coefficient", measuring the extent to which temporal dependence within the time series creates violations of exchangeability. Our characterization of the coverage probability is sharp over the class of stationary, $β$-mixing processes. Along the way, we introduce tools that may prove useful in analyzing other predictive inference methods for dependent data.

Predictive inference for time series: why is split conformal effective despite temporal dependence?

TL;DR

The paper develops a theoretical framework to explain why split conformal prediction remains effective for time series with temporal dependence and memory-based predictors. It introduces the switch coefficient to quantify deviations from exchangeability caused by dependence and shows that coverage loss for pretrained and split conformal methods can be bounded by this coefficient, with sharp results for stationary -mixing processes. The authors derive matching lower bounds, establish conditions preventing undercovering or overcovering, and extend the results to memory-enabled score functions, demonstrating faster (linear) rates compared to prior -type bounds. Overall, the work provides a principled, quantitative explanation for the empirical success of split conformal in dependent data settings and offers tools potentially applicable to broader non-exchangeable predictive inference methods.

Abstract

We consider the problem of uncertainty quantification for prediction in a time series: if we use past data to forecast the next time point, can we provide valid prediction intervals around our forecasts? To avoid placing distributional assumptions on the data, in recent years the conformal prediction method has been a popular approach for predictive inference, since it provides distribution-free coverage for any iid or exchangeable data distribution. However, in the time series setting, the strong empirical performance of conformal prediction methods is not well understood, since even short-range temporal dependence is a strong violation of the exchangeability assumption. Using predictors with "memory" -- i.e., predictors that utilize past observations, such as autoregressive models -- further exacerbates this problem. In this work, we examine the theoretical properties of split conformal prediction in the time series setting, including the case where predictors may have memory. Our results bound the loss of coverage of these methods in terms of a new "switch coefficient", measuring the extent to which temporal dependence within the time series creates violations of exchangeability. Our characterization of the coverage probability is sharp over the class of stationary, -mixing processes. Along the way, we introduce tools that may prove useful in analyzing other predictive inference methods for dependent data.

Paper Structure

This paper contains 28 sections, 11 theorems, 95 equations, 2 figures.

Table of Contents

  1. Introduction
  2. The predictive inference problem
  3. A motivating numerical experiment
  4. Pretrained and split conformal for predictors with memory
  5. Related work
  6. Existing results explaining conformal prediction on time series.
  7. Modifying conformal methods for the time series setting.
  8. Contributions and organization
  9. Quantifying dependence in the time series
  10. The switch coefficients
  11. Connection to mixing coefficients
  12. Switching data points, or switching scores?
  13. Main results
  14. Coverage guarantee for the pretrained setting
  15. A matching lower bound
  16. ...and 13 more sections

Key Result

Proposition 1

Suppose $\mathbf{Z}\in\mathcal{Z}^{n+1}$ is a stationary time series, with $\beta$-mixing coefficient $\beta(\tau)$. Then we have the following bound on the switch coefficients of $\mathbf{Z}$:

Figures (2)

  • Figure 1: Coverage of the pretrained conformal prediction set \ref{['eqn:C_pretrained']} on a sequence of length $n$ from the moving average process \ref{['eq:MAR']} of order $t$. The desired (i.e. nominal) coverage is $90\%$ throughout, and is denoted by a dotted line in all plots. Each point is generated by averaging over $10^6$ empirical trials.
  • Figure 2: Illustration of the definition of the subvectors $\Delta^0_{k,\tau}(\mathbf{w})$ (top) and $\Delta^1_{k,\tau}(\mathbf{w})$ (bottom), for a vector $\mathbf{w}$ of length $m=10$, in the case $k=3$, $\tau = 5$.

Theorems & Definitions (13)

  • Definition 1: The deletion operation
  • Definition 2: The switch coefficients
  • Proposition 1
  • Proposition 2
  • Theorem 1
  • Corollary 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Proposition 3
  • ...and 3 more