Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case

TL;DR

This work introduces a Transformer-based framework for time-series forecasting applied to influenza-like illness (ILI) data. It demonstrates that self-attention can capture long-range temporal dependencies and achieves strong performance against ARIMA and other deep baselines, including state-of-the-art results for US-level ILI forecasting. The study also investigates time-delay embeddings to approximate phase-space dynamics, showing robustness across univariate and multivariate inputs and suggesting extensions to spatio-temporal forecasting. Overall, the approach presents a general, extensible method for non-linear dynamical time-series forecasting with practical public-health applications.

Abstract

In this paper, we present a new approach to time series forecasting. Time series data are prevalent in many scientific and engineering disciplines. Time series forecasting is a crucial task in modeling time series data, and is an important area of machine learning. In this work we developed a novel method that employs Transformer-based machine learning models to forecast time series data. This approach works by leveraging self-attention mechanisms to learn complex patterns and dynamics from time series data. Moreover, it is a generic framework and can be applied to univariate and multivariate time series data, as well as time series embeddings. Using influenza-like illness (ILI) forecasting as a case study, we show that the forecasting results produced by our approach are favorably comparable to the state-of-the-art.

Figures (7)

• Figure 1: Architecture of Transformer-based forecasting model.
• Figure 2: Folded and unfolded representations of recurrent neural network.
• Figure 3: Long Short-Term Memory network and LSTM unit.
• Figure 4: Sequence-to-sequence (Seq2Seq) architecture to model sequence input and output.
• Figure 5: Using sliding window to construct supervised learning examples from time series data.