Progressive Neural Network for Multi-Horizon Time Series Forecasting

TL;DR

This paper introduces ProNet, an novel deep learning approach designed for multi-horizon time series forecasting, adaptively blending autoregressive (AR) and non-autoregressive (NAR) strategies, which outperforms state-of-the-art AR and NAR forecasting models.

Abstract

In this paper, we introduce ProNet, an novel deep learning approach designed for multi-horizon time series forecasting, adaptively blending autoregressive (AR) and non-autoregressive (NAR) strategies. Our method involves dividing the forecasting horizon into segments, predicting the most crucial steps in each segment non-autoregressively, and the remaining steps autoregressively. The segmentation process relies on latent variables, which effectively capture the significance of individual time steps through variational inference. In comparison to AR models, ProNet showcases remarkable advantages, requiring fewer AR iterations, resulting in faster prediction speed, and mitigating error accumulation. On the other hand, when compared to NAR models, ProNet takes into account the interdependency of predictions in the output space, leading to improved forecasting accuracy. Our comprehensive evaluation, encompassing four large datasets, and an ablation study, demonstrate the effectiveness of ProNet, highlighting its superior performance in terms of accuracy and prediction speed, outperforming state-of-the-art AR and NAR forecasting models.

Figures (8)

• Figure 1: Illustration of AR, ProNet partially AR and NAR decoding process: 1) AR decoder forecasts with covariates and all previous predictions; 2) NAR decoder forecasts all steps with covariates only in parallel; 3) our partially AR decoder divides horizon into segments (indicated by red dash lines), individual each segment is predicted autoregressively with covariates and previous predictions of all segments, while each prediction of segments can be made simultaneously.
• Figure 2: Partial correlation of Sanyo set for two different days (20 time steps for each day).
• Figure 3: Structure of the four components in ProNet: encoder, decoder, prior model $p_{\theta}$ and posterior model $q_{\phi}$.
• Figure 4: Creation process of progressive mask $M$: initial $M$ (upper left), $M$ after the 1st (upper right) and 2nd (lower left) iteration, and the final $M$ (lower right) when the forecasting horizon $T_h=7$, the segment size $n_g=3$ and starting positions of each segments $S_1=[1,3,5]$. We mark their changes in bold.
• Figure 5: Visualization of latent variable $z$: (a) original $z$, (b) re-weighted $z$. Higher brightness indicates the higher value of $z$ element.