Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improved AutoEncoder with LSTM module and KL divergence

Wei Huang, Bingyang Zhang, Kaituo Zhang, Hua Gao, Rongchun Wan

TL;DR

Experimental results show that the proposed Improved AutoEncoder with LSTM module and Kullback–Leibler divergence (IAE-LSTM-KL) model yields higher detection accuracy for anomalies and demonstrates enhanced robustness to contaminated outliers in the dataset.

Abstract

The task of anomaly detection is to separate anomalous data from normal data in the dataset. Models such as deep convolutional autoencoder (CAE) network and deep supporting vector data description (SVDD) model have been universally employed and have demonstrated significant success in detecting anomalies. However, the over-reconstruction ability of CAE network for anomalous data can easily lead to high false negative rate in detecting anomalous data. On the other hand, the deep SVDD model has the drawback of feature collapse, which leads to a decrease of detection accuracy for anomalies. To address these problems, we propose the Improved AutoEncoder with LSTM module and Kullback-Leibler divergence (IAE-LSTM-KL) model in this paper. An LSTM network is added after the encoder to memorize feature representations of normal data. In the meanwhile, the phenomenon of feature collapse can also be mitigated by penalizing the featured input to SVDD module via KL divergence. The efficacy of the IAE-LSTM-KL model is validated through experiments on both synthetic and real-world datasets. Experimental results show that IAE-LSTM-KL model yields higher detection accuracy for anomalies. In addition, it is also found that the IAE-LSTM-KL model demonstrates enhanced robustness to contaminated outliers in the dataset. All code may be found at https://github.com/crazyn2/IAE-LSTM-KL_codes

Improved AutoEncoder with LSTM module and KL divergence

TL;DR

Experimental results show that the proposed Improved AutoEncoder with LSTM module and Kullback–Leibler divergence (IAE-LSTM-KL) model yields higher detection accuracy for anomalies and demonstrates enhanced robustness to contaminated outliers in the dataset.

Abstract

The task of anomaly detection is to separate anomalous data from normal data in the dataset. Models such as deep convolutional autoencoder (CAE) network and deep supporting vector data description (SVDD) model have been universally employed and have demonstrated significant success in detecting anomalies. However, the over-reconstruction ability of CAE network for anomalous data can easily lead to high false negative rate in detecting anomalous data. On the other hand, the deep SVDD model has the drawback of feature collapse, which leads to a decrease of detection accuracy for anomalies. To address these problems, we propose the Improved AutoEncoder with LSTM module and Kullback-Leibler divergence (IAE-LSTM-KL) model in this paper. An LSTM network is added after the encoder to memorize feature representations of normal data. In the meanwhile, the phenomenon of feature collapse can also be mitigated by penalizing the featured input to SVDD module via KL divergence. The efficacy of the IAE-LSTM-KL model is validated through experiments on both synthetic and real-world datasets. Experimental results show that IAE-LSTM-KL model yields higher detection accuracy for anomalies. In addition, it is also found that the IAE-LSTM-KL model demonstrates enhanced robustness to contaminated outliers in the dataset. All code may be found at https://github.com/crazyn2/IAE-LSTM-KL_codes
Paper Structure (14 sections, 17 equations, 5 figures, 7 tables)

This paper contains 14 sections, 17 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Convolutional AE (CAE)
  4. Deep SVDD
  5. Improved AutoEncoder for Anomaly Detection (IAEAD)
  6. Proposed Method
  7. Framework of IAE-LSTM-KL model
  8. Introduction for LSTM module
  9. Detailed introduction for the IAE-LSTM-KL model
  10. Experiments
  11. Datasets
  12. Results
  13. Ablation Studies
  14. Conclusion

Figures (5)

  • Figure 1: (a). Schematic representation of the whole IAE-LSTM-KL model. (b). The role of KL divergence in forcing latent feature vectors following standard Gaussian distribution.
  • Figure 2: Demonstration of cell unit in the LSTM network.
  • Figure 3: Illustration of data smoothing based on sliding window.
  • Figure 4: AUROC values versus epochs in the training stage on CIFAR10 ((a) and (b)) and Fashion MNIST ((c) and (d)) datasets. The normal class in the dataset is shown above each subfigure. The training set only contains normal samples.
  • Figure 5: Relationship between AUROC values and anomaly ratio on the training set. (a). CIFAR10 dataset. (b). Fashion MNIST dataset.