Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transformer-based Multivariate Time Series Anomaly Localization

Charalampos Shimillas, Kleanthis Malialis, Konstantinos Fokianos, Marios M. Polycarpou

TL;DR

This work addresses the challenge of localizing anomalies in multivariate time series from CPS/IoT data by introducing a transformer-based framework that couples representation learning with two localization scores. The Space-Time Anomaly Score (STAS) leverages transformer latent representations to detect per-series anomalies while accounting for inter-series dependencies, and the Statistical Feature Anomaly Score (SFAS) provides a corrective signal using statistical features around anomalies. Together, STAS and SFAS enable accurate time-step, window-based, and segment-based localization, demonstrated to outperform state-of-the-art methods on synthetic and real-world datasets. The approach emphasizes unsupervised anomaly diagnosis, interpretability through attention-based analysis, and practical deployment considerations such as real-time scoring and CUSUM-based detection. This has potential impact for reliable CPS/IoT monitoring by enabling actionable, localized anomaly insights with robust performance across diverse data domains.

Abstract

With the growing complexity of Cyber-Physical Systems (CPS) and the integration of Internet of Things (IoT), the use of sensors for online monitoring generates large volume of multivariate time series (MTS) data. Consequently, the need for robust anomaly diagnosis in MTS is paramount to maintaining system reliability and safety. While significant advancements have been made in anomaly detection, localization remains a largely underexplored area, though crucial for intelligent decision-making. This paper introduces a novel transformer-based model for unsupervised anomaly diagnosis in MTS, with a focus on improving localization performance, through an in-depth analysis of the self-attention mechanism's learning behavior under both normal and anomalous conditions. We formulate the anomaly localization problem as a three-stage process: time-step, window, and segment-based. This leads to the development of the Space-Time Anomaly Score (STAS), a new metric inspired by the connection between transformer latent representations and space-time statistical models. STAS is designed to capture individual anomaly behaviors and inter-series dependencies, delivering enhanced localization performance. Additionally, the Statistical Feature Anomaly Score (SFAS) complements STAS by analyzing statistical features around anomalies, with their combination helping to reduce false alarms. Experiments on real world and synthetic datasets illustrate the model's superiority over state-of-the-art methods in both detection and localization tasks.

Transformer-based Multivariate Time Series Anomaly Localization

TL;DR

This work addresses the challenge of localizing anomalies in multivariate time series from CPS/IoT data by introducing a transformer-based framework that couples representation learning with two localization scores. The Space-Time Anomaly Score (STAS) leverages transformer latent representations to detect per-series anomalies while accounting for inter-series dependencies, and the Statistical Feature Anomaly Score (SFAS) provides a corrective signal using statistical features around anomalies. Together, STAS and SFAS enable accurate time-step, window-based, and segment-based localization, demonstrated to outperform state-of-the-art methods on synthetic and real-world datasets. The approach emphasizes unsupervised anomaly diagnosis, interpretability through attention-based analysis, and practical deployment considerations such as real-time scoring and CUSUM-based detection. This has potential impact for reliable CPS/IoT monitoring by enabling actionable, localized anomaly insights with robust performance across diverse data domains.

Abstract

With the growing complexity of Cyber-Physical Systems (CPS) and the integration of Internet of Things (IoT), the use of sensors for online monitoring generates large volume of multivariate time series (MTS) data. Consequently, the need for robust anomaly diagnosis in MTS is paramount to maintaining system reliability and safety. While significant advancements have been made in anomaly detection, localization remains a largely underexplored area, though crucial for intelligent decision-making. This paper introduces a novel transformer-based model for unsupervised anomaly diagnosis in MTS, with a focus on improving localization performance, through an in-depth analysis of the self-attention mechanism's learning behavior under both normal and anomalous conditions. We formulate the anomaly localization problem as a three-stage process: time-step, window, and segment-based. This leads to the development of the Space-Time Anomaly Score (STAS), a new metric inspired by the connection between transformer latent representations and space-time statistical models. STAS is designed to capture individual anomaly behaviors and inter-series dependencies, delivering enhanced localization performance. Additionally, the Statistical Feature Anomaly Score (SFAS) complements STAS by analyzing statistical features around anomalies, with their combination helping to reduce false alarms. Experiments on real world and synthetic datasets illustrate the model's superiority over state-of-the-art methods in both detection and localization tasks.
Paper Structure (20 sections, 16 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 20 sections, 16 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Problem formulation
  4. Anomaly diagnosis in MTS
  5. Preliminaries
  6. Analysis of representation learning of attention mechanism on MTS
  7. Unveiling hidden representation of MTS
  8. Effect of anomaly on the latent representation
  9. Proposed method
  10. Representation learning
  11. STAS
  12. SFAS
  13. Experimental setup
  14. Comparisons
  15. Datasets
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Subplots (a) and (c) show the reconstructions of Time Series 1 and Time Series 2 under normal conditions, with the two series being uncorrelated (Spearman, Pearson, and Kendall's Tau correlations all equal to zero). On the right, subplot (b) highlights anomalies in Time Series 1 (red boxes), while subplot (d) illustrates how these anomalies affect the reconstruction of the normal Time Series 2.
  • Figure 2: Overview of the proposed MTS anomaly localization method
  • Figure 3: Comparison of F1-scores using STAS, SFAS, and their combination for anomaly localization across the three datasets.
  • Figure 4: F1-scores of all methods across different window lengths for window-based localization, with arrows showing the maximum improvement over the best-performing method.

Theorems & Definitions (3)

  • Definition 1: Anomaly Detection
  • Definition 2: Time-step-wise Localization
  • Definition 3: Window-Based Localization