Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Selective Denoising Diffusion Model for Time Series Anomaly Detection

Kohei Obata, Zheng Chen, Yasuko Matsubara, Lingwei Zhu, Yasushi Sakurai

TL;DR

A novel diffusion-based method, named AnomalyFilter, which acts as a selective filter that only denoises anomaly parts in the instance while retaining normal parts, providing empirical support for its effectiveness in anomaly detection.

Abstract

Time series anomaly detection (TSAD) has been an important area of research for decades, with reconstruction-based methods, mostly based on generative models, gaining popularity and demonstrating success. Diffusion models have recently attracted attention due to their advanced generative capabilities. Existing diffusion-based methods for TSAD rely on a conditional strategy, which reconstructs input instances from white noise with the aid of the conditioner. However, this poses challenges in accurately reconstructing the normal parts, resulting in suboptimal detection performance. In response, we propose a novel diffusion-based method, named AnomalyFilter, which acts as a selective filter that only denoises anomaly parts in the instance while retaining normal parts. To build such a filter, we mask Gaussian noise during the training phase and conduct the denoising process without adding noise to the instances. The synergy of the two simple components greatly enhances the performance of naive diffusion models. Extensive experiments on five datasets demonstrate that AnomalyFilter achieves notably low reconstruction error on normal parts, providing empirical support for its effectiveness in anomaly detection. AnomalyFilter represents a pioneering approach that focuses on the noise design of diffusion models specifically tailored for TSAD.

Selective Denoising Diffusion Model for Time Series Anomaly Detection

TL;DR

A novel diffusion-based method, named AnomalyFilter, which acts as a selective filter that only denoises anomaly parts in the instance while retaining normal parts, providing empirical support for its effectiveness in anomaly detection.

Abstract

Time series anomaly detection (TSAD) has been an important area of research for decades, with reconstruction-based methods, mostly based on generative models, gaining popularity and demonstrating success. Diffusion models have recently attracted attention due to their advanced generative capabilities. Existing diffusion-based methods for TSAD rely on a conditional strategy, which reconstructs input instances from white noise with the aid of the conditioner. However, this poses challenges in accurately reconstructing the normal parts, resulting in suboptimal detection performance. In response, we propose a novel diffusion-based method, named AnomalyFilter, which acts as a selective filter that only denoises anomaly parts in the instance while retaining normal parts. To build such a filter, we mask Gaussian noise during the training phase and conduct the denoising process without adding noise to the instances. The synergy of the two simple components greatly enhances the performance of naive diffusion models. Extensive experiments on five datasets demonstrate that AnomalyFilter achieves notably low reconstruction error on normal parts, providing empirical support for its effectiveness in anomaly detection. AnomalyFilter represents a pioneering approach that focuses on the noise design of diffusion models specifically tailored for TSAD.
Paper Structure (62 sections, 1 theorem, 13 equations, 8 figures, 7 tables, 4 algorithms)

This paper contains 62 sections, 1 theorem, 13 equations, 8 figures, 7 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Results Overview
  3. Contributions
  4. Related Work
  5. Diffusion Model in Time Series
  6. Diffusion Model for Time Series Anomaly Detection
  7. Denoising Model for Anomaly Detection
  8. Preliminaries
  9. Problem Definition
  10. Denoising Diffusion Probablistic Model
  11. Forward Process
  12. Reverse Process
  13. Objective Function
  14. Anomaly Detection with Diffusion Model
  15. Training Phase
  16. ...and 47 more sections

Key Result

Proposition 1

Assume we have a noise $\epsilon_{t} = \bm{B}\circ\epsilon^{1} + (\bm{1}-\bm{B})\circ\epsilon^{2}$, where $\epsilon^{1}$ and $\epsilon^{2}$ are sampled from some distributions, and $\bm{B}$ is a mask with elements drawn from a Bernoulli distribution, $\bm{B}_{k,l} \sim Bernoulli(p)$. Then, the loss

Figures (8)

  • Figure 1: Reconstruction examples of encoder-decoder models (i.e., BeatGAN and Anomaly-Transformer), conditional diffusion models (i.e., IMDiffusion), and proposed AnomalyFilter.
  • Figure 2: An overview of AnomalyFilter framework. The model is trained to predict masked Gaussian noise, with the aim of learning to denoise the anomaly parts (i.e., nonmasked parts) and retaining the normal parts (i.e., masked parts). Noiseless inference operates without noise at the beginning and during the denoising steps. Thus, when given a normal instance, it outputs an instance with no change. When given an anomaly instance, only anomalous parts are removed. AnomalyFilter captures temporal and inter-variable dependencies by temporal and feature transformer layers and predicts noise added to the input.
  • Figure 3: Effect of loss weights and noise strength.
  • Figure 4: Critical difference diagram of VUS-PR.
  • Figure 5: Reconstruction examples across DDPM variants of $\#029$ ABP on UCR.
  • ...and 3 more figures

Theorems & Definitions (2)

  • Proposition 1
  • Proof 1