Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Considering Nonstationary within Multivariate Time Series with Variational Hierarchical Transformer for Forecasting

Muyao Wang, Wenchao Chen, Bo Chen

TL;DR

The paper tackles the forecasting of multivariate time series under non-stationarity and intrinsic stochasticity. It introduces HTV-Trans, a probabilistic dynamic model that fuses a Hierarchical Time Series Probabilistic Generative Module (HTPGM) with a Transformer to recover multi-scale non-stationary information, while employing a series stationarization and denormalization scheme. An autoencoding variational inference objective with a combined prediction and reconstruction loss guides training, enabling robust representation learning and better long-horizon forecasts. Empirical results on seven diverse datasets demonstrate that HTV-Trans outperforms state-of-the-art Transformer-based methods, validating the approach and highlighting its practical potential for real-world MTS forecasting where non-stationarity and stochasticity are prevalent.

Abstract

The forecasting of Multivariate Time Series (MTS) has long been an important but challenging task. Due to the non-stationary problem across long-distance time steps, previous studies primarily adopt stationarization method to attenuate the non-stationary problem of the original series for better predictability. However, existing methods always adopt the stationarized series, which ignores the inherent non-stationarity, and has difficulty in modeling MTS with complex distributions due to the lack of stochasticity. To tackle these problems, we first develop a powerful hierarchical probabilistic generative module to consider the non-stationarity and stochastic characteristics within MTS, and then combine it with transformer for a well-defined variational generative dynamic model named Hierarchical Time series Variational Transformer (HTV-Trans), which recovers the intrinsic non-stationary information into temporal dependencies. Being a powerful probabilistic model, HTV-Trans is utilized to learn expressive representations of MTS and applied to forecasting tasks. Extensive experiments on diverse datasets show the efficiency of HTV-Trans on MTS forecasting tasks

Considering Nonstationary within Multivariate Time Series with Variational Hierarchical Transformer for Forecasting

TL;DR

The paper tackles the forecasting of multivariate time series under non-stationarity and intrinsic stochasticity. It introduces HTV-Trans, a probabilistic dynamic model that fuses a Hierarchical Time Series Probabilistic Generative Module (HTPGM) with a Transformer to recover multi-scale non-stationary information, while employing a series stationarization and denormalization scheme. An autoencoding variational inference objective with a combined prediction and reconstruction loss guides training, enabling robust representation learning and better long-horizon forecasts. Empirical results on seven diverse datasets demonstrate that HTV-Trans outperforms state-of-the-art Transformer-based methods, validating the approach and highlighting its practical potential for real-world MTS forecasting where non-stationarity and stochasticity are prevalent.

Abstract

The forecasting of Multivariate Time Series (MTS) has long been an important but challenging task. Due to the non-stationary problem across long-distance time steps, previous studies primarily adopt stationarization method to attenuate the non-stationary problem of the original series for better predictability. However, existing methods always adopt the stationarized series, which ignores the inherent non-stationarity, and has difficulty in modeling MTS with complex distributions due to the lack of stochasticity. To tackle these problems, we first develop a powerful hierarchical probabilistic generative module to consider the non-stationarity and stochastic characteristics within MTS, and then combine it with transformer for a well-defined variational generative dynamic model named Hierarchical Time series Variational Transformer (HTV-Trans), which recovers the intrinsic non-stationary information into temporal dependencies. Being a powerful probabilistic model, HTV-Trans is utilized to learn expressive representations of MTS and applied to forecasting tasks. Extensive experiments on diverse datasets show the efficiency of HTV-Trans on MTS forecasting tasks
Paper Structure (15 sections, 7 equations, 6 figures, 2 tables)

This paper contains 15 sections, 7 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Backgrounds
  3. Multivariate Time Series Forecasting
  4. Non-stationary Problems of Time Series
  5. Method
  6. Problem Definition
  7. Time Series Stationarization
  8. Hierarchical Time Series Variational Transformer
  9. Model Training
  10. Experiments
  11. Datasets and Set Up
  12. Forecasting Main Results
  13. Ablation Study
  14. Balance between Stationary and Non-stationary Information
  15. Conclusion

Figures (6)

  • Figure 1: The temporal dependency, stochasticity and non-stationarity within MTS.
  • Figure 2: Graphical illustration of different operations of the HTV-Trans: (a) generative process of HTPGM, (b) the inference scheme of HTPGM. (c) the fusion of different scale information and stationarization input series for forecasting.
  • Figure 3: the whole framework of HTV-Trans.
  • Figure 4: Visualizations on ETTm2 dataset given by different models
  • Figure 5: The effectiveness evaluation of hierarchical architecture on ETTh1(left), ETTh2(middle) and ETTm1(right) dataset.
  • ...and 1 more figures