Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning

Bahareh Golchin, Banafsheh Rekabdar

TL;DR

The paper tackles anomaly detection in time series under scarce labeled data and evolving patterns. It introduces RLVAL, a framework that blends Deep Reinforcement Learning (via DQN), a Variational Autoencoder, and Active Learning with an LSTM backbone to model temporal dependencies. Key contributions include a VAE-driven intrinsic reward, a mixed extrinsic/intrinsic reward for efficient exploration, and a margin-sampling-based active-learning component, evaluated on Yahoo A1Benchmark and KPI with strong F1 gains using minimal labels (e.g., 0.834 at 1% and 0.921 at 10% on Yahoo; 0.825 at 0.05% and 0.908 at 0.1% on KPI). The approach advances data-efficient, adaptive time-series anomaly detection with clear practical impact for IT operations, data centers, and finance, and suggests future enhancement via integration with large language models.

Abstract

A novel approach to detecting anomalies in time series data is presented in this paper. This approach is pivotal in domains such as data centers, sensor networks, and finance. Traditional methods often struggle with manual parameter tuning and cannot adapt to new anomaly types. Our method overcomes these limitations by integrating Deep Reinforcement Learning (DRL) with a Variational Autoencoder (VAE) and Active Learning. By incorporating a Long Short-Term Memory (LSTM) network, our approach models sequential data and its dependencies effectively, allowing for the detection of new anomaly classes with minimal labeled data. Our innovative DRL- VAE and Active Learning combination significantly improves existing methods, as shown by our evaluations on real-world datasets, enhancing anomaly detection techniques and advancing time series analysis.

Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning

TL;DR

The paper tackles anomaly detection in time series under scarce labeled data and evolving patterns. It introduces RLVAL, a framework that blends Deep Reinforcement Learning (via DQN), a Variational Autoencoder, and Active Learning with an LSTM backbone to model temporal dependencies. Key contributions include a VAE-driven intrinsic reward, a mixed extrinsic/intrinsic reward for efficient exploration, and a margin-sampling-based active-learning component, evaluated on Yahoo A1Benchmark and KPI with strong F1 gains using minimal labels (e.g., 0.834 at 1% and 0.921 at 10% on Yahoo; 0.825 at 0.05% and 0.908 at 0.1% on KPI). The approach advances data-efficient, adaptive time-series anomaly detection with clear practical impact for IT operations, data centers, and finance, and suggests future enhancement via integration with large language models.

Abstract

A novel approach to detecting anomalies in time series data is presented in this paper. This approach is pivotal in domains such as data centers, sensor networks, and finance. Traditional methods often struggle with manual parameter tuning and cannot adapt to new anomaly types. Our method overcomes these limitations by integrating Deep Reinforcement Learning (DRL) with a Variational Autoencoder (VAE) and Active Learning. By incorporating a Long Short-Term Memory (LSTM) network, our approach models sequential data and its dependencies effectively, allowing for the detection of new anomaly classes with minimal labeled data. Our innovative DRL- VAE and Active Learning combination significantly improves existing methods, as shown by our evaluations on real-world datasets, enhancing anomaly detection techniques and advancing time series analysis.

Paper Structure

This paper contains 20 sections, 13 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Statistical-based Methods
  4. Machine Learning-based Methods
  5. Background
  6. Reinforcement Learning in Anomaly Detection
  7. Deep Q-Networks and Q-Learning
  8. Variational Autoencoder
  9. Active Learning in Machine Learning Systems
  10. PROPOSED METHOD
  11. Anomaly Detection with Variational Autoencoders
  12. Deep Reinforcement Learning Framework
  13. Active Learning
  14. Experiments
  15. Datasets
  16. ...and 5 more sections

Figures (3)

  • Figure 1: Overview of RLVAL system
  • Figure 2: The VAE framework
  • Figure 3: Sample time windows from the A1Benchmark dataset demonstrating how the RL model processes inputs to perform actions and receive rewards.