Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TransNAS-TSAD: Harnessing Transformers for Multi-Objective Neural Architecture Search in Time Series Anomaly Detection

Ijaz Ul Haq, Byung Suk Lee, Donna M. Rizzo

TL;DR

TransNAS-TSAD tackles time series anomaly detection by uniting transformer architectures with neural architecture search (NAS) optimized via NSGA-II to balance detection accuracy and computational efficiency. The framework introduces a multi-objective NAS process, an offline optimization phase, and a three-phase adversarial enhancement to improve robustness across diverse univariate and multivariate datasets. It defines and employs the Efficiency-Accuracy-Complexity Score (EACS) to evaluate models in terms of F1 performance, training time, and parameter count, and demonstrates strong results across benchmarks such as NAB, UCR, MBA, SMAP, SWaT, WADI, and SMD. The work provides a practical, adaptable solution for real-time TSAD and offers a pathway for future enhancements in dynamic thresholding, online learning, and human-in-the-loop design, with explicit Pareto-front guidance for deployment under varying resource constraints.

Abstract

The surge in real-time data collection across various industries has underscored the need for advanced anomaly detection in both univariate and multivariate time series data. This paper introduces TransNAS-TSAD, a framework that synergizes the transformer architecture with neural architecture search (NAS), enhanced through NSGA-II algorithm optimization. This approach effectively tackles the complexities of time series data, balancing computational efficiency with detection accuracy. Our evaluation reveals that TransNAS-TSAD surpasses conventional anomaly detection models due to its tailored architectural adaptability and the efficient exploration of complex search spaces, leading to marked improvements in diverse data scenarios. We also introduce the Efficiency-Accuracy-Complexity Score (EACS) as a new metric for assessing model performance, emphasizing the balance between accuracy and computational resources. TransNAS-TSAD sets a new benchmark in time series anomaly detection, offering a versatile, efficient solution for complex real-world applications. This research highlights the TransNAS-TSAD potential across a wide range of industry applications and paves the way for future developments in the field.

TransNAS-TSAD: Harnessing Transformers for Multi-Objective Neural Architecture Search in Time Series Anomaly Detection

TL;DR

TransNAS-TSAD tackles time series anomaly detection by uniting transformer architectures with neural architecture search (NAS) optimized via NSGA-II to balance detection accuracy and computational efficiency. The framework introduces a multi-objective NAS process, an offline optimization phase, and a three-phase adversarial enhancement to improve robustness across diverse univariate and multivariate datasets. It defines and employs the Efficiency-Accuracy-Complexity Score (EACS) to evaluate models in terms of F1 performance, training time, and parameter count, and demonstrates strong results across benchmarks such as NAB, UCR, MBA, SMAP, SWaT, WADI, and SMD. The work provides a practical, adaptable solution for real-time TSAD and offers a pathway for future enhancements in dynamic thresholding, online learning, and human-in-the-loop design, with explicit Pareto-front guidance for deployment under varying resource constraints.

Abstract

The surge in real-time data collection across various industries has underscored the need for advanced anomaly detection in both univariate and multivariate time series data. This paper introduces TransNAS-TSAD, a framework that synergizes the transformer architecture with neural architecture search (NAS), enhanced through NSGA-II algorithm optimization. This approach effectively tackles the complexities of time series data, balancing computational efficiency with detection accuracy. Our evaluation reveals that TransNAS-TSAD surpasses conventional anomaly detection models due to its tailored architectural adaptability and the efficient exploration of complex search spaces, leading to marked improvements in diverse data scenarios. We also introduce the Efficiency-Accuracy-Complexity Score (EACS) as a new metric for assessing model performance, emphasizing the balance between accuracy and computational resources. TransNAS-TSAD sets a new benchmark in time series anomaly detection, offering a versatile, efficient solution for complex real-world applications. This research highlights the TransNAS-TSAD potential across a wide range of industry applications and paves the way for future developments in the field.
Paper Structure (45 sections, 17 equations, 4 figures, 5 tables, 1 algorithm)

This paper contains 45 sections, 17 equations, 4 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Problem Definition
  5. Multi-Objective Transformer Model Optimization
  6. Data Processing and Model Training
  7. Anomaly Scoring and Thresholding
  8. TransNAS-TSAD Transformer Architecture
  9. Data Augmentation Mechanisms Within the Model:
  10. Encoder Operations
  11. Decoder Operations
  12. Neural Architecture Search (NAS) Framework for TransNAS-TSAD
  13. Search Space Definition
  14. TransNAS-TSAD Evaluation Strategy
  15. Multi-Objective Optimization with NSGA-II
  16. ...and 30 more sections

Figures (4)

  • Figure 1: Workflow of the TransNAS-TSAD Process for Time Series Anomaly Detection using Multi-Objective Neural Architecture Search (NAS) with NSGA-II Optimization.
  • Figure 2: Anomaly detection in TransNAS-TSAD on dimension 0 of the SMD test dataset, depicting true versus predicted values and identified anomalies
  • Figure 3: Analysis of hyperparameter importance for F1 score optimization in TransNAS-TSAD across four datasets (NAB, MBA, SMAP, and WADI). The plots illustrate the relative impact of different hyperparameters on the F1 score, guiding the model's fine-tuning process for effective anomaly detection in diverse time series datasets
  • Figure 4: Pareto front plots illustrating the trade-off between F1 score and number of parameters for NAB, MBA, SMAP, and WADI datasets in the TransNAS-TSAD optimization process.