Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

Guokun Lai, Wei-Cheng Chang, Yiming Yang, Hanxiao Liu

TL;DR

The paper addresses multivariate time series forecasting with both short-term and long-term patterns by introducing LSTNet, a hybrid architecture that combines CNN for local patterns, GRU-based RNN for longer dependencies, a recurrent-skip mechanism for periodicity, a temporal attention layer, and an autoregressive component for scale robustness. It provides a detailed formulation of the model, its components, and training objective, followed by extensive experiments on four real-world datasets showing superior performance over strong baselines, especially on data with clear periodicity. Ablation studies validate the contribution of each component, particularly the AR module, and demonstrate LSTNet's ability to model mixtures of daily and weekly patterns. The work offers a practical, robust approach for complex time series forecasting with potential impact in energy, transportation, and finance domains, and suggests directions for automatic pattern-aware adaptations and attribute integration.

Abstract

Multivariate time series forecasting is an important machine learning problem across many domains, including predictions of solar plant energy output, electricity consumption, and traffic jam situation. Temporal data arise in these real-world applications often involves a mixture of long-term and short-term patterns, for which traditional approaches such as Autoregressive models and Gaussian Process may fail. In this paper, we proposed a novel deep learning framework, namely Long- and Short-term Time-series network (LSTNet), to address this open challenge. LSTNet uses the Convolution Neural Network (CNN) and the Recurrent Neural Network (RNN) to extract short-term local dependency patterns among variables and to discover long-term patterns for time series trends. Furthermore, we leverage traditional autoregressive model to tackle the scale insensitive problem of the neural network model. In our evaluation on real-world data with complex mixtures of repetitive patterns, LSTNet achieved significant performance improvements over that of several state-of-the-art baseline methods. All the data and experiment codes are available online.

Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

TL;DR

The paper addresses multivariate time series forecasting with both short-term and long-term patterns by introducing LSTNet, a hybrid architecture that combines CNN for local patterns, GRU-based RNN for longer dependencies, a recurrent-skip mechanism for periodicity, a temporal attention layer, and an autoregressive component for scale robustness. It provides a detailed formulation of the model, its components, and training objective, followed by extensive experiments on four real-world datasets showing superior performance over strong baselines, especially on data with clear periodicity. Ablation studies validate the contribution of each component, particularly the AR module, and demonstrate LSTNet's ability to model mixtures of daily and weekly patterns. The work offers a practical, robust approach for complex time series forecasting with potential impact in energy, transportation, and finance domains, and suggests directions for automatic pattern-aware adaptations and attribute integration.

Abstract

Multivariate time series forecasting is an important machine learning problem across many domains, including predictions of solar plant energy output, electricity consumption, and traffic jam situation. Temporal data arise in these real-world applications often involves a mixture of long-term and short-term patterns, for which traditional approaches such as Autoregressive models and Gaussian Process may fail. In this paper, we proposed a novel deep learning framework, namely Long- and Short-term Time-series network (LSTNet), to address this open challenge. LSTNet uses the Convolution Neural Network (CNN) and the Recurrent Neural Network (RNN) to extract short-term local dependency patterns among variables and to discover long-term patterns for time series trends. Furthermore, we leverage traditional autoregressive model to tackle the scale insensitive problem of the neural network model. In our evaluation on real-world data with complex mixtures of repetitive patterns, LSTNet achieved significant performance improvements over that of several state-of-the-art baseline methods. All the data and experiment codes are available online.

Paper Structure

This paper contains 20 sections, 14 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Background
  3. Framework
  4. Problem Formulation
  5. Convolutional Component
  6. Recurrent Component
  7. Recurrent-skip Component
  8. Temporal Attention Layer
  9. Autoregressive Component
  10. Objective function
  11. Optimization Strategy
  12. Evaluation
  13. Methods for Comparison
  14. Metrics
  15. Data
  16. ...and 5 more sections

Figures (7)

  • Figure 1: The hourly occupancy rate of a road in the bay area for 2 weeks
  • Figure 2: An overview of the Long- and Short-term Time-series network (LSTNet)
  • Figure 3: Autocorrelation graphs of sampled variables form four datasets.
  • Figure 4: Simulation Test: Left side is the training set and right side is test set.
  • Figure 5: Results of LSTNet in the ablation tests on the Solar-Energy, Traffic and Electricity dataset
  • ...and 2 more figures