Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transformer Conformal Prediction for Time Series

Junghwan Lee, Chen Xu, Yao Xie

TL;DR

This work addresses uncertainty quantification for time series under non-exchangeability by integrating a Transformer-based conditional quantile estimator into Sequential Predictive Conformal Inference (SPCI), producing valid prediction intervals with reduced width. The proposed SPCI-T learns temporal dependencies among past residuals and features by using a decoder-only Transformer to estimate quantiles of future residuals $\hat{\epsilon}_t$, with training via a quantile loss $\mathcal{L}(\hat{\epsilon}, \hat{\epsilon}', p)$. Empirical results on simulated and real datasets show that SPCI-T achieves narrower intervals while maintaining target coverage, outperforming SPCI, EnbPI, and NexCP, and benefiting further from additional time features. The method offers a practical advancement for reliable uncertainty quantification and multi-step forecasting in real-world time-series applications like energy and weather data.

Abstract

We present a conformal prediction method for time series using the Transformer architecture to capture long-memory and long-range dependencies. Specifically, we use the Transformer decoder as a conditional quantile estimator to predict the quantiles of prediction residuals, which are used to estimate the prediction interval. We hypothesize that the Transformer decoder benefits the estimation of the prediction interval by learning temporal dependencies across past prediction residuals. Our comprehensive experiments using simulated and real data empirically demonstrate the superiority of the proposed method compared to the existing state-of-the-art conformal prediction methods.

Transformer Conformal Prediction for Time Series

TL;DR

This work addresses uncertainty quantification for time series under non-exchangeability by integrating a Transformer-based conditional quantile estimator into Sequential Predictive Conformal Inference (SPCI), producing valid prediction intervals with reduced width. The proposed SPCI-T learns temporal dependencies among past residuals and features by using a decoder-only Transformer to estimate quantiles of future residuals , with training via a quantile loss . Empirical results on simulated and real datasets show that SPCI-T achieves narrower intervals while maintaining target coverage, outperforming SPCI, EnbPI, and NexCP, and benefiting further from additional time features. The method offers a practical advancement for reliable uncertainty quantification and multi-step forecasting in real-world time-series applications like energy and weather data.

Abstract

We present a conformal prediction method for time series using the Transformer architecture to capture long-memory and long-range dependencies. Specifically, we use the Transformer decoder as a conditional quantile estimator to predict the quantiles of prediction residuals, which are used to estimate the prediction interval. We hypothesize that the Transformer decoder benefits the estimation of the prediction interval by learning temporal dependencies across past prediction residuals. Our comprehensive experiments using simulated and real data empirically demonstrate the superiority of the proposed method compared to the existing state-of-the-art conformal prediction methods.
Paper Structure (23 sections, 12 equations, 2 figures, 7 tables)

This paper contains 23 sections, 12 equations, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Problem setup
  3. Method
  4. Sequential Predictive Conformal Inference
  5. Conformal Prediction for Time Series with Transformer
  6. Experiments
  7. Setup
  8. Simulation
  9. Dataset
  10. Results
  11. Real data Experiments
  12. Dataset
  13. Results
  14. Conclusion
  15. Additional Details of Experiments
  16. ...and 8 more sections

Figures (2)

  • Figure 1: Visual description of our proposed method. Prediction residuals ($\hat{\epsilon}_t$) and corresponding features ($X_t$) are used as input to a stack of Transformer decoder layers to predict the quantiles of future residual(s), which are used to estimate prediction intervals.
  • Figure 2: Prediction intervals estimated by SPCI-T and baselines on non-stationary simulated dataset.