Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Better Batch for Deep Probabilistic Time Series Forecasting

Vincent Zhihao Zheng, Seongjin Choi, Lijun Sun

TL;DR

This work proposes an innovative training method that incorporates error autocorrelation to enhance probabilistic forecasting accuracy and improves the performance of two different neural forecasting models across a range of datasets.

Abstract

Deep probabilistic time series forecasting has gained attention for its ability to provide nonlinear approximation and valuable uncertainty quantification for decision-making. However, existing models often oversimplify the problem by assuming a time-independent error process and overlooking serial correlation. To overcome this limitation, we propose an innovative training method that incorporates error autocorrelation to enhance probabilistic forecasting accuracy. Our method constructs a mini-batch as a collection of $D$ consecutive time series segments for model training. It explicitly learns a time-varying covariance matrix over each mini-batch, encoding error correlation among adjacent time steps. The learned covariance matrix can be used to improve prediction accuracy and enhance uncertainty quantification. We evaluate our method on two different neural forecasting models and multiple public datasets. Experimental results confirm the effectiveness of the proposed approach in improving the performance of both models across a range of datasets, resulting in notable improvements in predictive accuracy.

Better Batch for Deep Probabilistic Time Series Forecasting

TL;DR

This work proposes an innovative training method that incorporates error autocorrelation to enhance probabilistic forecasting accuracy and improves the performance of two different neural forecasting models across a range of datasets.

Abstract

Deep probabilistic time series forecasting has gained attention for its ability to provide nonlinear approximation and valuable uncertainty quantification for decision-making. However, existing models often oversimplify the problem by assuming a time-independent error process and overlooking serial correlation. To overcome this limitation, we propose an innovative training method that incorporates error autocorrelation to enhance probabilistic forecasting accuracy. Our method constructs a mini-batch as a collection of consecutive time series segments for model training. It explicitly learns a time-varying covariance matrix over each mini-batch, encoding error correlation among adjacent time steps. The learned covariance matrix can be used to improve prediction accuracy and enhance uncertainty quantification. We evaluate our method on two different neural forecasting models and multiple public datasets. Experimental results confirm the effectiveness of the proposed approach in improving the performance of both models across a range of datasets, resulting in notable improvements in predictive accuracy.
Paper Structure (16 sections, 14 equations, 3 figures, 3 tables)

This paper contains 16 sections, 14 equations, 3 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. PRELIMINARIES
  3. Probabilistic Time Series Forecasting
  4. Error Autocorrelation
  5. RELATED WORK
  6. Probabilistic Time Series Forecasting
  7. Modeling Correlated Errors
  8. OUR METHOD
  9. Training with Mini-batch
  10. Multi-step Rolling Prediction
  11. EXPERIMENTS
  12. Datasets and Models
  13. Evaluation against Baseline
  14. Interpretation of Correlation
  15. Computational Cost
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Autocorrelation Function (ACF) of the One-step-ahead Prediction Residuals. The results depict the prediction outcomes generated by DeepAR for two time series in the $\mathtt{m4\_hourly}$ dataset. The shaded area indicates the 95% confidence interval, highlighting regions where the correlation is statistically insignificant.
  • Figure 2: Example of a Mini-batch. Only one-step-ahead prediction is involved during training.
  • Figure 3: (a) Component weights for generating the correlation matrix of an example time series from the $\mathtt{m4\_hourly}$ dataset. Parameters $w_0, w_1, w_2$ represent the component weights for kernel matrices associated with lengthscales $l=1,2,3$, respectively, and $w_3$ is the component weight for the identity matrix. Shaded areas distinguish different days; (b) and (d): The autocorrelation function (ACF) indicated by the resulting error correlation matrix $\boldsymbol{C}_t$ at 6:00 and 16:00. Given the rapid decay of the ACF, we only plot 24 lags to enhance clarity in our visualization; (c) and (e): The corresponding covariance matrix of the associated target variables $\boldsymbol{\Sigma}_t^{\text{bat}}=\mathop{\mathrm{diag}}\nolimits(\boldsymbol{\sigma}_t^{\text{bat}})\boldsymbol{C}_t\mathop{\mathrm{diag}}\nolimits(\boldsymbol{\sigma}_t^{\text{bat}})$ at 6:00 and 16:00, respectively.