Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data

Shreshth Tuli, Giuliano Casale, Nicholas R. Jennings

TL;DR

TranAD tackles anomaly detection and diagnosis in high-dimensional time series with a transformer-based architecture that uses focus-score self-conditioning and offline two-phase adversarial training. The method integrates MAML for rapid adaptation with limited data and employs dynamic POT thresholds for robust online detection. Empirical results across nine public benchmarks show TranAD outperforms state-of-the-art baselines in both detection and diagnosis, with substantial reductions in training time. The work demonstrates the practical impact of fast, accurate anomaly forecasting and root-cause analysis in industrial systems, and provides a public implementation for broader adoption.

Abstract

Efficient anomaly detection and diagnosis in multivariate time-series data is of great importance for modern industrial applications. However, building a system that is able to quickly and accurately pinpoint anomalous observations is a challenging problem. This is due to the lack of anomaly labels, high data volatility and the demands of ultra-low inference times in modern applications. Despite the recent developments of deep learning approaches for anomaly detection, only a few of them can address all of these challenges. In this paper, we propose TranAD, a deep transformer network based anomaly detection and diagnosis model which uses attention-based sequence encoders to swiftly perform inference with the knowledge of the broader temporal trends in the data. TranAD uses focus score-based self-conditioning to enable robust multi-modal feature extraction and adversarial training to gain stability. Additionally, model-agnostic meta learning (MAML) allows us to train the model using limited data. Extensive empirical studies on six publicly available datasets demonstrate that TranAD can outperform state-of-the-art baseline methods in detection and diagnosis performance with data and time-efficient training. Specifically, TranAD increases F1 scores by up to 17%, reducing training times by up to 99% compared to the baselines.

TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data

TL;DR

TranAD tackles anomaly detection and diagnosis in high-dimensional time series with a transformer-based architecture that uses focus-score self-conditioning and offline two-phase adversarial training. The method integrates MAML for rapid adaptation with limited data and employs dynamic POT thresholds for robust online detection. Empirical results across nine public benchmarks show TranAD outperforms state-of-the-art baselines in both detection and diagnosis, with substantial reductions in training time. The work demonstrates the practical impact of fast, accurate anomaly forecasting and root-cause analysis in industrial systems, and provides a public implementation for broader adoption.

Abstract

Efficient anomaly detection and diagnosis in multivariate time-series data is of great importance for modern industrial applications. However, building a system that is able to quickly and accurately pinpoint anomalous observations is a challenging problem. This is due to the lack of anomaly labels, high data volatility and the demands of ultra-low inference times in modern applications. Despite the recent developments of deep learning approaches for anomaly detection, only a few of them can address all of these challenges. In this paper, we propose TranAD, a deep transformer network based anomaly detection and diagnosis model which uses attention-based sequence encoders to swiftly perform inference with the knowledge of the broader temporal trends in the data. TranAD uses focus score-based self-conditioning to enable robust multi-modal feature extraction and adversarial training to gain stability. Additionally, model-agnostic meta learning (MAML) allows us to train the model using limited data. Extensive empirical studies on six publicly available datasets demonstrate that TranAD can outperform state-of-the-art baseline methods in detection and diagnosis performance with data and time-efficient training. Specifically, TranAD increases F1 scores by up to 17%, reducing training times by up to 99% compared to the baselines.
Paper Structure (20 sections, 16 equations, 7 figures, 7 tables, 2 algorithms)

This paper contains 20 sections, 16 equations, 7 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Problem Formulation
  5. Data Preprocessing
  6. Transformer Model
  7. Offline Two-Phase Adversarial Training
  8. Online Inference, Anomaly Detection and Diagnosis
  9. Experiments
  10. Datasets
  11. Evaluation Metrics
  12. Anomaly Detection
  13. Anomaly Diagnosis
  14. Results
  15. Analyses
  16. ...and 5 more sections

Figures (7)

  • Figure 1: The TranAD Model.
  • Figure 2: Visualization of anomaly prediction.
  • Figure 3: Visualization of focus and attention scores.
  • Figure 4: Critical difference diagrams for F1 and AUC scores using the Wilkoxon pairwised signed rank test (with $\alpha=0.05$) on all datasets. Rightmost methods are ranked higher.
  • Figure 5: Predicted and Ground Truth labels for the MSDS test set using the TranAD model.
  • ...and 2 more figures