Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mamba Adaptive Anomaly Transformer with association discrepancy for time series

Abdellah Zakaria Sellam, Ilyes Benaissa, Abdelmalik Taleb-Ahmed, Luigi Patrono, Cosimo Distante

TL;DR

The paper tackles unsupervised anomaly detection in time series by integrating Sparse Attention with a Mamba-Selective State Space Model (Mamba-SSM) and adaptive gating to improve reconstruction and localization. It introduces Anomaly Sparse Attention and an adaptive MAAT block that fuse local and long-range dependencies via skip connections and gating, guided by Association Discrepancy. Empirical results across eight real-world datasets show MAAT consistently outperforms the Anomaly Transformer and DCdetector in F1, precision, and recall, demonstrating robustness in noisy, non-stationary environments. The work offers a scalable, efficient framework for real-world time-series anomaly detection with potential for online adaptation and hybrid learning extensions.

Abstract

Anomaly detection in time series is essential for industrial monitoring and environmental sensing, yet distinguishing anomalies from complex patterns remains challenging. Existing methods like the Anomaly Transformer and DCdetector have progressed, but they face limitations such as sensitivity to short-term contexts and inefficiency in noisy, non-stationary environments. To overcome these issues, we introduce MAAT, an improved architecture that enhances association discrepancy modeling and reconstruction quality. MAAT features Sparse Attention, efficiently capturing long-range dependencies by focusing on relevant time steps, thereby reducing computational redundancy. Additionally, a Mamba-Selective State Space Model is incorporated into the reconstruction module, utilizing a skip connection and Gated Attention to improve anomaly localization and detection performance. Extensive experiments show that MAAT significantly outperforms previous methods, achieving better anomaly distinguishability and generalization across various time series applications, setting a new standard for unsupervised time series anomaly detection in real-world scenarios.

Mamba Adaptive Anomaly Transformer with association discrepancy for time series

TL;DR

The paper tackles unsupervised anomaly detection in time series by integrating Sparse Attention with a Mamba-Selective State Space Model (Mamba-SSM) and adaptive gating to improve reconstruction and localization. It introduces Anomaly Sparse Attention and an adaptive MAAT block that fuse local and long-range dependencies via skip connections and gating, guided by Association Discrepancy. Empirical results across eight real-world datasets show MAAT consistently outperforms the Anomaly Transformer and DCdetector in F1, precision, and recall, demonstrating robustness in noisy, non-stationary environments. The work offers a scalable, efficient framework for real-world time-series anomaly detection with potential for online adaptation and hybrid learning extensions.

Abstract

Anomaly detection in time series is essential for industrial monitoring and environmental sensing, yet distinguishing anomalies from complex patterns remains challenging. Existing methods like the Anomaly Transformer and DCdetector have progressed, but they face limitations such as sensitivity to short-term contexts and inefficiency in noisy, non-stationary environments. To overcome these issues, we introduce MAAT, an improved architecture that enhances association discrepancy modeling and reconstruction quality. MAAT features Sparse Attention, efficiently capturing long-range dependencies by focusing on relevant time steps, thereby reducing computational redundancy. Additionally, a Mamba-Selective State Space Model is incorporated into the reconstruction module, utilizing a skip connection and Gated Attention to improve anomaly localization and detection performance. Extensive experiments show that MAAT significantly outperforms previous methods, achieving better anomaly distinguishability and generalization across various time series applications, setting a new standard for unsupervised time series anomaly detection in real-world scenarios.

Paper Structure

This paper contains 39 sections, 17 equations, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Methodology
  4. Background knowledge
  5. Association Discrepancy
  6. Minimize Phase:
  7. Association-based Anomaly Criterion
  8. Sparse Attention for Efficient Sequence Attentiong
  9. Mamba State Space Model
  10. Mamba Adaptive Anomaly Transformer Architecture
  11. Anomaly sparse attention
  12. MAAT Adaptive Block
  13. Adaptive Gating and Reconstruction:
  14. Experiments
  15. Benchmark Datasets
  16. ...and 24 more sections

Figures (2)

  • Figure 1: The figure illustrates the MAAT framework for time series anomaly detection. Block A (Anomaly Sparse Attention Module) computes prior and series associations using sparse Attention and a learnable Gaussian kernel to model temporal dependencies. Block B (Reconstruction Module) refines the feature representations using layer normalization, feedforward processing, and MAAT blocks to reconstruct input signals effectively. Block C represents the MAAT Block that integrates the Mamba state-space model to capture long-range dependencies, followed by a gated attention mechanism that adaptively fuses the reconstructed output.
  • Figure 2: Comparison of reconstruction loss between our model and the Anomaly Transformer across different datasets. Left column: Reconstruction loss curves for both models. Right column: Difference in reconstruction loss.