A Memory-Network Based Solution for Multivariate Time-Series Forecasting

TL;DR

Multivariate time-series forecasting faces challenges in modeling long-range dependencies and cross-variable interactions while maintaining interpretability. The authors present MTNet, a memory-augmented network with a large memory module, three encoders, and an autoregressive component, enabling attention over historical chunks and capturing periodic patterns. The approach combines a neural predictor with a linear AR term and uses block-level attention to reveal which memory segments influence predictions, achieving state-of-the-art results on six public datasets for both univariate and multivariate forecasting. The work also demonstrates the interpretability of MTNet through memory attention visualizations, with potential for practical deployment across domains like finance, traffic, and energy.

Abstract

Multivariate time series forecasting is extensively studied throughout the years with ubiquitous applications in areas such as finance, traffic, environment, etc. Still, concerns have been raised on traditional methods for incapable of modeling complex patterns or dependencies lying in real word data. To address such concerns, various deep learning models, mainly Recurrent Neural Network (RNN) based methods, are proposed. Nevertheless, capturing extremely long-term patterns while effectively incorporating information from other variables remains a challenge for time-series forecasting. Furthermore, lack-of-explainability remains one serious drawback for deep neural network models. Inspired by Memory Network proposed for solving the question-answering task, we propose a deep learning based model named Memory Time-series network (MTNet) for time series forecasting. MTNet consists of a large memory component, three separate encoders, and an autoregressive component to train jointly. Additionally, the attention mechanism designed enable MTNet to be highly interpretable. We can easily tell which part of the historic data is referenced the most.

Figures (4)

• Figure 1: An overview of Memory Time-series network (MTNet) on the right and the details of the encoder architecture on the left
• Figure 2: Plot of the attention weights between Input and Memory component for MTNet.
• Figure 3: Prediction results of DA-RNN and MTNet GEFCom2014 Electricity Price dataset visualized. Segmnets are randomly sampled from the testing set.
• Figure 4: Prediction results of LSTNet and MTNet on Traffic dataset with horizon 24 visualized. Segments are randomly sampled from the testing set