Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GAS-Norm: Score-Driven Adaptive Normalization for Non-Stationary Time Series Forecasting in Deep Learning

Edoardo Urettini, Daniele Atzeni, Reshawn J. Ramjattan, Antonio Carta

TL;DR

GAS-Norm is introduced, a novel methodology for adaptive time series normalization and forecasting based on the combination of a Generalized Autoregressive Score (GAS) model and a Deep Neural Network that improves the performance of the model in non-stationary settings.

Abstract

Despite their popularity, deep neural networks (DNNs) applied to time series forecasting often fail to beat simpler statistical models. One of the main causes of this suboptimal performance is the data non-stationarity present in many processes. In particular, changes in the mean and variance of the input data can disrupt the predictive capability of a DNN. In this paper, we first show how DNN forecasting models fail in simple non-stationary settings. We then introduce GAS-Norm, a novel methodology for adaptive time series normalization and forecasting based on the combination of a Generalized Autoregressive Score (GAS) model and a Deep Neural Network. The GAS approach encompasses a score-driven family of models that estimate the mean and variance at each new observation, providing updated statistics to normalize the input data of the deep model. The output of the DNN is eventually denormalized using the statistics forecasted by the GAS model, resulting in a hybrid approach that leverages the strengths of both statistical modeling and deep learning. The adaptive normalization improves the performance of the model in non-stationary settings. The proposed approach is model-agnostic and can be applied to any DNN forecasting model. To empirically validate our proposal, we first compare GAS-Norm with other state-of-the-art normalization methods. We then combine it with state-of-the-art DNN forecasting models and test them on real-world datasets from the Monash open-access forecasting repository. Results show that deep forecasting models improve their performance in 21 out of 25 settings when combined with GAS-Norm compared to other normalization methods.

GAS-Norm: Score-Driven Adaptive Normalization for Non-Stationary Time Series Forecasting in Deep Learning

TL;DR

GAS-Norm is introduced, a novel methodology for adaptive time series normalization and forecasting based on the combination of a Generalized Autoregressive Score (GAS) model and a Deep Neural Network that improves the performance of the model in non-stationary settings.

Abstract

Despite their popularity, deep neural networks (DNNs) applied to time series forecasting often fail to beat simpler statistical models. One of the main causes of this suboptimal performance is the data non-stationarity present in many processes. In particular, changes in the mean and variance of the input data can disrupt the predictive capability of a DNN. In this paper, we first show how DNN forecasting models fail in simple non-stationary settings. We then introduce GAS-Norm, a novel methodology for adaptive time series normalization and forecasting based on the combination of a Generalized Autoregressive Score (GAS) model and a Deep Neural Network. The GAS approach encompasses a score-driven family of models that estimate the mean and variance at each new observation, providing updated statistics to normalize the input data of the deep model. The output of the DNN is eventually denormalized using the statistics forecasted by the GAS model, resulting in a hybrid approach that leverages the strengths of both statistical modeling and deep learning. The adaptive normalization improves the performance of the model in non-stationary settings. The proposed approach is model-agnostic and can be applied to any DNN forecasting model. To empirically validate our proposal, we first compare GAS-Norm with other state-of-the-art normalization methods. We then combine it with state-of-the-art DNN forecasting models and test them on real-world datasets from the Monash open-access forecasting repository. Results show that deep forecasting models improve their performance in 21 out of 25 settings when combined with GAS-Norm compared to other normalization methods.
Paper Structure (15 sections, 24 equations, 4 figures, 4 tables)

This paper contains 15 sections, 24 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Proposed Method
  4. Why Adaptive Normalization for a DNN?
  5. Parameter Filtering for Non-stationary Time Series
  6. GAS-Norm
  7. Experiments with Normalization Methods
  8. Quantitative Evaluation
  9. Experiments with SOTA Forecasting Models
  10. Results
  11. Conclusion
  12. Appendix
  13. Data Statistics and Hyperparameters
  14. Gaussian Score Driven Adaptive Normalization
  15. Student's t Score Driven Adaptive Normalization

Figures (4)

  • Figure 1: (a) The evolving coordinates of a 3D Lorenz attractor. (b) The ratio between the MSE of the nonlinear MLP and the MSE of the linear model. The green area is where the nonlinear model is better. (c) Prediction of the 500-steps ahead value done by the linear and the nonlinear model. The vertical line shows the end of the training data.
  • Figure 2: GAS-Norm procedure for a single time step input. Statistical parameters $\theta$ in this case are the means and variances.
  • Figure 3: Comparison of the filtering process of GAS-Norm with different normalization strengths ($\gamma$). The shaded area is the mean $\pm$ standard deviation.
  • Figure 4: Examples of input/output sequences with different methods' filtered and predicted statistics together with the denormalized prediction of the DNN. The top row shows a sequence from AR data, while the bottom row shows from the VIX data with an outlier. The red vertical dashed line splits the context part of the sequence from the part to be predicted.