Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TSCMamba: Mamba Meets Multi-View Learning for Time Series Classification

Md Atik Ahamed, Qiang Cheng

TL;DR

TSCMamba tackles multivariate time series classification by explicitly addressing shift equivariance and inversion invariance. It combines spectral features from continuous wavelet transforms with local ROCKET-based and global MLP-based temporal features, fused through a learnable gating mechanism, and processed by a Mamba state-space model with a novel tango-scanning scheme to model long-range dependencies efficiently. The approach achieves state-of-the-art performance across 30 diverse UEA datasets, while reducing floating point operations and memory usage relative to strong baselines. The work introduces inversion-invariant learning and provides thorough ablations, complexity analyses, and practical guidance, highlighting the potential of multi-view learning and efficient state-space models for real-world time-series tasks.

Abstract

Multivariate time series classification (TSC) is critical for various applications in fields such as healthcare and finance. While various approaches for TSC have been explored, important properties of time series, such as shift equivariance and inversion invariance, are largely underexplored by existing works. To fill this gap, we propose a novel multi-view approach to capture patterns with properties like shift equivariance. Our method integrates diverse features, including spectral, temporal, local, and global features, to obtain rich, complementary contexts for TSC. We use continuous wavelet transform to capture time-frequency features that remain consistent even when the input is shifted in time. These features are fused with temporal convolutional or multilayer perceptron features to provide complex local and global contextual information. We utilize the Mamba state space model for efficient and scalable sequence modeling and capturing long-range dependencies in time series. Moreover, we introduce a new scanning scheme for Mamba, called tango scanning, to effectively model sequence relationships and leverage inversion invariance, thereby enhancing our model's generalization and robustness. Experiments on two sets of benchmark datasets (10+20 datasets) demonstrate our approach's effectiveness, achieving average accuracy improvements of 4.01-6.45\% and 7.93\% respectively, over leading TSC models such as TimesNet and TSLANet.

TSCMamba: Mamba Meets Multi-View Learning for Time Series Classification

TL;DR

TSCMamba tackles multivariate time series classification by explicitly addressing shift equivariance and inversion invariance. It combines spectral features from continuous wavelet transforms with local ROCKET-based and global MLP-based temporal features, fused through a learnable gating mechanism, and processed by a Mamba state-space model with a novel tango-scanning scheme to model long-range dependencies efficiently. The approach achieves state-of-the-art performance across 30 diverse UEA datasets, while reducing floating point operations and memory usage relative to strong baselines. The work introduces inversion-invariant learning and provides thorough ablations, complexity analyses, and practical guidance, highlighting the potential of multi-view learning and efficient state-space models for real-world time-series tasks.

Abstract

Multivariate time series classification (TSC) is critical for various applications in fields such as healthcare and finance. While various approaches for TSC have been explored, important properties of time series, such as shift equivariance and inversion invariance, are largely underexplored by existing works. To fill this gap, we propose a novel multi-view approach to capture patterns with properties like shift equivariance. Our method integrates diverse features, including spectral, temporal, local, and global features, to obtain rich, complementary contexts for TSC. We use continuous wavelet transform to capture time-frequency features that remain consistent even when the input is shifted in time. These features are fused with temporal convolutional or multilayer perceptron features to provide complex local and global contextual information. We utilize the Mamba state space model for efficient and scalable sequence modeling and capturing long-range dependencies in time series. Moreover, we introduce a new scanning scheme for Mamba, called tango scanning, to effectively model sequence relationships and leverage inversion invariance, thereby enhancing our model's generalization and robustness. Experiments on two sets of benchmark datasets (10+20 datasets) demonstrate our approach's effectiveness, achieving average accuracy improvements of 4.01-6.45\% and 7.93\% respectively, over leading TSC models such as TimesNet and TSLANet.
Paper Structure (34 sections, 1 theorem, 17 equations, 14 figures, 5 tables, 2 algorithms)

This paper contains 34 sections, 1 theorem, 17 equations, 14 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Traditional Methods
  4. Deep Learning Approaches
  5. Transformer-Based Methods
  6. State-Space Models
  7. Methodology
  8. Spectral Representation
  9. Temporal Feature Extraction
  10. Fusing Multi-View Representations
  11. Inferring with Time-Channel Tango Scanning
  12. Output Class Representation
  13. Explanation of Our Scanning Mechanism
  14. Experiments and Result Analysis
  15. Datasets
  16. ...and 19 more sections

Key Result

Theorem 1

(Gu and Dao 2023) When $N = 1, A = -1, B = 1$, $s_{\Delta} = Linear(x)$, and $\tau_{\Delta} = softplus$, then the selective SSM recurrence takes the form of where $g_k$ is the gate.

Figures (14)

  • Figure 1: Schematic illustration of the proposed model TSCMamba. This diagram illustrates the architecture of our approach, featuring tango scanning. Here, DP refers to Depth-wise Pooling (DP) and LP refers to Linear Projection (LP). A switch gate selectively activates the utilization of either ROCKET or MLP-derived features. The MLP module, depicted in the bottom right, comprises two layers with an optional dropout mechanism interspersed for regularization. Here, $B$ = Batch, $L$ = Length, $D$ = Dimension.
  • Figure 2: Regular forward scanning process. The arrow from node $a$ to node $b$ means $a$ depends on $b$.
  • Figure 3: Flipped sequence scanning process.
  • Figure 4: Conceptual illustration of the tango scanning mechanism for enhancing inversion invariance in TSC.
  • Figure 5: Attention matrix visualization for different scanning protocols, illustrating the attended regions. Tango scanning achieves the maximum possible pairwise attention coverage with one Mamba.
  • ...and 9 more figures

Theorems & Definitions (1)

  • Theorem 1