Stabilizing Adversarially Learned One-Class Novelty Detection Using Pseudo Anomalies

TL;DR

A robust anomaly detection framework that overcomes instability by transforming the fundamental role of the discriminator from identifying real vs. fake data to distinguishing good vs. bad quality reconstructions is proposed.

Abstract

Recently, anomaly scores have been formulated using reconstruction loss of the adversarially learned generators and/or classification loss of discriminators. Unavailability of anomaly examples in the training data makes optimization of such networks challenging. Attributed to the adversarial training, performance of such models fluctuates drastically with each training step, making it difficult to halt the training at an optimal point. In the current study, we propose a robust anomaly detection framework that overcomes such instability by transforming the fundamental role of the discriminator from identifying real vs. fake data to distinguishing good vs. bad quality reconstructions. For this purpose, we propose a method that utilizes the current state as well as an old state of the same generator to create good and bad quality reconstruction examples. The discriminator is trained on these examples to detect the subtle distortions that are often present in the reconstructions of anomalous data. In addition, we propose an efficient generic criterion to stop the training of our model, ensuring elevated performance. Extensive experiments performed on six datasets across multiple domains including image and video based anomaly detection, medical diagnosis, and network security, have demonstrated excellent performance of our approach.

Figures (11)

• Figure 1: a) An illustration of the behavior of a conventional discriminator, when used as it is for outlier detection problem. As it is trained to discriminate real from fake, the decision boundary lies in between the real inliers and fake inliers. b) The desideratum is to train a discriminator that can identify the reconstructions of inliers from the reconstructions of outliers.
• Figure 2: Dynamics of AUC performance over training epochs: The baseline ($\mathcal{G}$+$\mathcal{D}$) shows fluctuations while our approach not only shows stability across various epochs but also yields higher AUC.
• Figure 3: Our proposed adversarially learned novelty detection framework. Phase one is the baseline training, carried out to achieve a reasonably evolved state of $\mathcal{G}$ and $\mathcal{D}$. Two states of $\mathcal{G}$, a less evolved $\mathcal{G}_{o}$ and a more evolved $\mathcal{G}_{n}$, are frozen during this phase. Then, in phase two, only $\mathcal{D}$ is trained to distinguish between good and bad quality reconstruction examples. Good quality examples correspond to real training images as well as the training images reconstructed through $\mathcal{G}$. Whereas, bad quality examples are created using $\mathcal{G}_{o}$ as well as the newly introduced pseudo anomaly module. This module assists $\mathcal{D}$ in learning the underlying patterns of the anomalous input reconstructions. At test time, only $\mathcal{G}_n$ and $\mathcal{D}$ are used for inference and the output of $\mathcal{D}$ is directly considered as anomaly score.
• Figure 4: The three variants of our proposed pseudo anomaly module for generating pseudo anomalies $\hat{P}_{o}$. (a) in Early fusion, two input data instances are averaged and passed through $\mathcal{G}_{o}$ to obtain $\hat{P}_{o}$. (b) in Late fusion, two data instances are reconstructed separately using $\mathcal{G}_{o}$ and the outputs are averaged to form $\hat{P}_{o}$. (c) in Latent fusion, latent embeddings of the two input instances are computed using the encoder $\mathcal{G}_{o}^E$ of $\mathcal{G}_{o}$ and averaged. The resultant embedding is then passed through the decoder $\mathcal{G}^D_{o}$ of $\mathcal{G}_{o}$ to obtain $\hat{P}_{o}$.
• Figure 5: Example images taken from different intermediate steps of our framework. (a) Left to right: normal input images ($X$), high quality reconstructions ($\hat{X}_n$), low quality reconstructions ($\hat{X}_{o}$), pseudo anomalies ($\hat{P}_o$), pseudo anomaly reconstructions ($\hat{P}_{n}$). (b) Left column shows outlier/anomaly test images whereas right column shows respective reconstructions $\mathcal{G}_n$($X$).