Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Treating Brain-inspired Memories as Priors for Diffusion Model to Forecast Multivariate Time Series

Muyao Wang, Wenchao Chen, Zhibin Duan, Bo Chen

TL;DR

This work tackles multivariate time series forecasting under limited past context by introducing a brain-inspired memory-augmented diffusion model (Bim-Diff). It pairwise merges semantic memory for general, recurring patterns with episodic memory for rare, event-like patterns, both shared across channels, as priors to a conditional diffusion framework guiding future predictions. Through memory recall/update mechanisms and a memory-conditioned denoising objective, the approach achieves state-of-the-art performance across eight real-world datasets and demonstrates improved efficiency via a simple MLP backbone and DDIM-based sampling. The memory modules also provide interpretable cross-channel correlations, enabling more robust and scalable forecasting in heterogeneous MTS settings.

Abstract

Forecasting Multivariate Time Series (MTS) involves significant challenges in various application domains. One immediate challenge is modeling temporal patterns with the finite length of the input. These temporal patterns usually involve periodic and sudden events that recur across different channels. To better capture temporal patterns, we get inspiration from humans' memory mechanisms and propose a channel-shared, brain-inspired memory module for MTS. Specifically, brain-inspired memory comprises semantic and episodic memory, where the former is used to capture general patterns, such as periodic events, and the latter is employed to capture special patterns, such as sudden events, respectively. Meanwhile, we design corresponding recall and update mechanisms to better utilize these patterns. Furthermore, acknowledging the capacity of diffusion models to leverage memory as a prior, we present a brain-inspired memory-augmented diffusion model. This innovative model retrieves relevant memories for different channels, utilizing them as distinct priors for MTS predictions. This incorporation significantly enhances the accuracy and robustness of predictions. Experimental results on eight datasets consistently validate the superiority of our approach in capturing and leveraging diverse recurrent temporal patterns across different channels.

Treating Brain-inspired Memories as Priors for Diffusion Model to Forecast Multivariate Time Series

TL;DR

This work tackles multivariate time series forecasting under limited past context by introducing a brain-inspired memory-augmented diffusion model (Bim-Diff). It pairwise merges semantic memory for general, recurring patterns with episodic memory for rare, event-like patterns, both shared across channels, as priors to a conditional diffusion framework guiding future predictions. Through memory recall/update mechanisms and a memory-conditioned denoising objective, the approach achieves state-of-the-art performance across eight real-world datasets and demonstrates improved efficiency via a simple MLP backbone and DDIM-based sampling. The memory modules also provide interpretable cross-channel correlations, enabling more robust and scalable forecasting in heterogeneous MTS settings.

Abstract

Forecasting Multivariate Time Series (MTS) involves significant challenges in various application domains. One immediate challenge is modeling temporal patterns with the finite length of the input. These temporal patterns usually involve periodic and sudden events that recur across different channels. To better capture temporal patterns, we get inspiration from humans' memory mechanisms and propose a channel-shared, brain-inspired memory module for MTS. Specifically, brain-inspired memory comprises semantic and episodic memory, where the former is used to capture general patterns, such as periodic events, and the latter is employed to capture special patterns, such as sudden events, respectively. Meanwhile, we design corresponding recall and update mechanisms to better utilize these patterns. Furthermore, acknowledging the capacity of diffusion models to leverage memory as a prior, we present a brain-inspired memory-augmented diffusion model. This innovative model retrieves relevant memories for different channels, utilizing them as distinct priors for MTS predictions. This incorporation significantly enhances the accuracy and robustness of predictions. Experimental results on eight datasets consistently validate the superiority of our approach in capturing and leveraging diverse recurrent temporal patterns across different channels.
Paper Structure (28 sections, 18 equations, 6 figures, 3 tables, 1 algorithm)

This paper contains 28 sections, 18 equations, 6 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Memory Augmented Diffusion Model
  4. Problem Definition
  5. Denoising Diffusion Probabilistic Models
  6. Memory-augmented Conditional Diffusion Models for Time Series Prediction
  7. Capturing Temporal Patterns via Brain-inspired Memory
  8. Semantic Memory
  9. Episodic Memory
  10. Model Training
  11. Experiments
  12. Baselines and Experimental Settings
  13. Main Results
  14. Ablation Study
  15. Analysis of Recurrent Patterns and Channel Correlations
  16. ...and 13 more sections

Figures (6)

  • Figure 1: The red box represents the current MTS forecasting task. The blue box and green box represent diverse temporal patterns across different channels which can help channel 3 and 4 to forecast better. Because those temporal patterns have happened in other channels in history.
  • Figure 2: The framework of brain-inspired memory-augmented diffusion model.
  • Figure 3: The visualization of input series, similar score matrix of semantic memory and episodic memory. The input series contains 7 channels. The size of the episodic memory is set to 70, and the size of the semantic memory is set to 64.
  • Figure 4: (a)-(b): The effect of the size of semantic (a) and episodic (b) memory on ETTh1 dataset. (c): Inference time (in ms) of various time series diffusion models with different prediction horizons (H) on the ETTh1(other method results come from shen2024multiresolution).
  • Figure 5: The details of episodic memory update strategy.
  • ...and 1 more figures