Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeMa: Dual-Path Delay-Aware Mamba for Efficient Multivariate Time Series Analysis

Rui An, Haohao Qu, Wenqi Fan, Xuequn Shang, Qing Li

TL;DR

DeMa addresses scalable, accurate multivariate time series analysis by decoupling intra-series temporal dynamics from inter-series cross-variate interactions using a dual-path back­bone. It introduces Mamba-SSD for temporal modeling and Mamba-DALA for delay-aware cross-variate attention, integrated through DuoMNet blocks and an Adaptive Fourier Filter to separate spectral content. The approach achieves state-of-the-art results across forecasting, imputation, anomaly detection, and classification tasks while maintaining linear-time complexity $O(2NLd^2)$ per block, enabling efficient training on long horizons and large-scale MTS. The work demonstrates strong practical impact for real-world, high-dimensional MTS applications by delivering both accuracy gains and substantial efficiency improvements over Transformer-based and prior Mamba-based methods.

Abstract

Accurate and efficient multivariate time series (MTS) analysis is increasingly critical for a wide range of intelligent applications. Within this realm, Transformers have emerged as the predominant architecture due to their strong ability to capture pairwise dependencies. However, Transformer-based models suffer from quadratic computational complexity and high memory overhead, limiting their scalability and practical deployment in long-term and large-scale MTS modeling. Recently, Mamba has emerged as a promising linear-time alternative with high expressiveness. Nevertheless, directly applying vanilla Mamba to MTS remains suboptimal due to three key limitations: (i) the lack of explicit cross-variate modeling, (ii) difficulty in disentangling the entangled intra-series temporal dynamics and inter-series interactions, and (iii) insufficient modeling of latent time-lag interaction effects. These issues constrain its effectiveness across diverse MTS tasks. To address these challenges, we propose DeMa, a dual-path delay-aware Mamba backbone. DeMa preserves Mamba's linear-complexity advantage while substantially improving its suitability for MTS settings. Specifically, DeMa introduces three key innovations: (i) it decomposes the MTS into intra-series temporal dynamics and inter-series interactions; (ii) it develops a temporal path with a Mamba-SSD module to capture long-range dynamics within each individual series, enabling series-independent, parallel computation; and (iii) it designs a variate path with a Mamba-DALA module that integrates delay-aware linear attention to model cross-variate dependencies. Extensive experiments on five representative tasks, long- and short-term forecasting, data imputation, anomaly detection, and series classification, demonstrate that DeMa achieves state-of-the-art performance while delivering remarkable computational efficiency.

DeMa: Dual-Path Delay-Aware Mamba for Efficient Multivariate Time Series Analysis

TL;DR

DeMa addresses scalable, accurate multivariate time series analysis by decoupling intra-series temporal dynamics from inter-series cross-variate interactions using a dual-path back­bone. It introduces Mamba-SSD for temporal modeling and Mamba-DALA for delay-aware cross-variate attention, integrated through DuoMNet blocks and an Adaptive Fourier Filter to separate spectral content. The approach achieves state-of-the-art results across forecasting, imputation, anomaly detection, and classification tasks while maintaining linear-time complexity per block, enabling efficient training on long horizons and large-scale MTS. The work demonstrates strong practical impact for real-world, high-dimensional MTS applications by delivering both accuracy gains and substantial efficiency improvements over Transformer-based and prior Mamba-based methods.

Abstract

Accurate and efficient multivariate time series (MTS) analysis is increasingly critical for a wide range of intelligent applications. Within this realm, Transformers have emerged as the predominant architecture due to their strong ability to capture pairwise dependencies. However, Transformer-based models suffer from quadratic computational complexity and high memory overhead, limiting their scalability and practical deployment in long-term and large-scale MTS modeling. Recently, Mamba has emerged as a promising linear-time alternative with high expressiveness. Nevertheless, directly applying vanilla Mamba to MTS remains suboptimal due to three key limitations: (i) the lack of explicit cross-variate modeling, (ii) difficulty in disentangling the entangled intra-series temporal dynamics and inter-series interactions, and (iii) insufficient modeling of latent time-lag interaction effects. These issues constrain its effectiveness across diverse MTS tasks. To address these challenges, we propose DeMa, a dual-path delay-aware Mamba backbone. DeMa preserves Mamba's linear-complexity advantage while substantially improving its suitability for MTS settings. Specifically, DeMa introduces three key innovations: (i) it decomposes the MTS into intra-series temporal dynamics and inter-series interactions; (ii) it develops a temporal path with a Mamba-SSD module to capture long-range dynamics within each individual series, enabling series-independent, parallel computation; and (iii) it designs a variate path with a Mamba-DALA module that integrates delay-aware linear attention to model cross-variate dependencies. Extensive experiments on five representative tasks, long- and short-term forecasting, data imputation, anomaly detection, and series classification, demonstrate that DeMa achieves state-of-the-art performance while delivering remarkable computational efficiency.
Paper Structure (34 sections, 1 theorem, 31 equations, 6 figures, 5 tables, 1 algorithm)

This paper contains 34 sections, 1 theorem, 31 equations, 6 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notations and Definitions
  4. Mamba
  5. State Space Model (SSM)
  6. Selective State Space Model (Mamba)
  7. Mamba-2
  8. The Proposed Method: DeMa
  9. Structure Overview
  10. Adaptive Fourier Filter
  11. DuoMNet Block
  12. Cross-Time Scan and Cross-Variate Scan
  13. Cross-Time Dependencies Modeling via Mamba-SSD
  14. Cross-Variate Dependencies Modeling via Mamba-DALA
  15. DuoMNet Block Design
  16. ...and 19 more sections

Key Result

Theorem 1

Given the input series $\mathcal{X}=\widetilde{\mathcal{X}}+\hat{\mathcal{X}}$ and let $E=\{e_1,\dots,e_N\}$ be an orthogonal basis. Suppose and define the supports $\widetilde{E}=\{e_i\in E:\ a_i\neq 0\}$ and $\hat{E}=\{e_i\in E:\ b_i\neq 0\}$. If $\widetilde{\mathcal{X}}$ and $\hat{\mathcal{X}}$ are not orthogonal, i.e., $\langle \widetilde{\mathcal{X}}, \hat{\mathcal{X}}\rangle \neq 0$, then $

Figures (6)

  • Figure 1: Dependency modeling strategies and computational complexity of representative MTS architectures. (a) Tokenization and induced dependency patterns (variate-mixing, variate-independent, variate-dependent). (b) Complexity comparison of representative models. Here, $T$ is the lookback length, $N$ the number of variates, $L$ the token length, and $d$ the embedding dimension, in typical long-horizon settings, $T>L\gg N,d$. DeMa decouples temporal modeling and delay-aware cross-variate interactions with linear-time complexity.
  • Figure 2: The overall framework of DeMa. The proposed DeMa (Left) comprises three key components: Adaptive Fourier Filter and stacked DuoMNet Blocks, and Task-specific Projection. Most importantly, the DuoMNet Block (Right) integrates insights from the Mamba-SSD path and the Mamba-DALA path to achieve comprehensive modeling of cross-time and cross-variate dependencies.
  • Figure 3: Performance comparison on classification and anomaly detection tasks. Results are averaged over multiple datasets; higher is better.
  • Figure 4: Model performance comparison (left) and efficiency comparison (right). DeMa achieves consistent state-of-the-art performance across five mainstream time-series tasks. Meanwhile, DeMa attains the lowest mean squared error while maintaining shorter training time and a smaller memory footprint.
  • Figure 5: Efficiency analysis of GPU memory and running time in a long-term lookback-window scenario. Our proposed DeMa scales linearly with the series length, whereas the vanilla Transformer exhibits quadratic complexity with respect to the series length.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Theorem 1: Non-orthogonality implies basis overlap
  • proof