Improved Anomaly Detection through Conditional Latent Space VAE Ensembles

TL;DR

The proposed novel Conditional Latent space Variational Autoencoder (CL-VAE) shows increased benefits from ensembling, a more interpretable latent space, and an increased ability to learn patterns in complex data with limited model sizes.

Abstract

We propose a novel Conditional Latent space Variational Autoencoder (CL-VAE) to perform improved pre-processing for anomaly detection on data with known inlier classes and unknown outlier classes. This proposed variational autoencoder (VAE) improves latent space separation by conditioning on information within the data. The method fits a unique prior distribution to each class in the dataset, effectively expanding the classic prior distribution for VAEs to include a Gaussian mixture model. An ensemble of these VAEs are merged in the latent spaces to form a group consensus that greatly improves the accuracy of anomaly detection across data sets. Our approach is compared against the capabilities of a typical VAE, a CNN, and a PCA, with regards AUC for anomaly detection. The proposed model shows increased accuracy in anomaly detection, achieving an AUC of 97.4% on the MNIST dataset compared to 95.7% for the second best model. In addition, the CL-VAE shows increased benefits from ensembling, a more interpretable latent space, and an increased ability to learn patterns in complex data with limited model sizes.

Figures (4)

• Figure 1: Schematic of a typical VAE. The encoder neural network maps the input $x$ to the latent space described by the Gaussian distribution $z$. Then the decoder neural network creates a reconstruction of the input in $\hat{x}$.
• Figure 2: Latent space representation for the three datasets (top: MNIST, center: Fashion-MNIST, bottom: CIFAR-10) using the regular VAE (two leftmost columns) and CL-VAE (two rightmost columns). For each model, two plots are presented; the training set with class centers marked with their corresponding digit, and the test set with anomalies and normal classes in different colors.
• Figure 3: Confidence intervals of the AUC anomaly detection as a function of ensemble size using the 5 different evaluation methods.
• Figure 4: Examples of misclassifications on the three datasets for each of the five evaluation methods.