Autoencoders for Anomaly Detection are Unreliable

TL;DR

This paper shows that autoencoders, including PCA-based, linear, and non-linear variants, can perfectly reconstruct anomalies far from training data, invalidating the common assumption that reconstruction loss separates normal from anomalous instances. Through theoretical proofs and extensive experiments on tabular and real-world image data (including MNIST), the authors demonstrate out-of-bounds reconstruction across activation functions (ReLU, sigmoid, and others) and architectures, highlighting safety implications for anomaly detection pipelines. The work provides a formal framework for anomaly reconstruction, reveals practical failure modes, and calls for validation strategies (e.g., adversarial latent-space searches) to ensure reliability in critical applications. Overall, the findings urge caution in using reconstruction loss as a sole anomaly score and motivate development of more robust detectors.

Abstract

Autoencoders are frequently used for anomaly detection, both in the unsupervised and semi-supervised settings. They rely on the assumption that when trained using the reconstruction loss, they will be able to reconstruct normal data more accurately than anomalous data. Some recent works have posited that this assumption may not always hold, but little has been done to study the validity of the assumption in theory. In this work we show that this assumption indeed does not hold, and illustrate that anomalies, lying far away from normal data, can be perfectly reconstructed in practice. We revisit the theory of failure of linear autoencoders for anomaly detection by showing how they can perfectly reconstruct out of bounds, or extrapolate undesirably, and note how this can be dangerous in safety critical applications. We connect this to non-linear autoencoders through experiments on both tabular data and real-world image data, the two primary application areas of autoencoders for anomaly detection.

Figures (2)

• Figure 1: Plots of the contours of the reconstruction loss of non-linear autoencoders when applied to 3 distinct datasets. The datasets consist out of a 100 samples from a 2D Gaussian (a, e), 100 samples for each of 2 different 2D Gaussians (b, f), and a 100 samples from a classic banana shaped function with Gaussian noise (c, d). In (a,b,c,e,f) a [2,5,1,5,2] autoencoder is used, while for (d) a deeper [2,100,20,1,20,100,2] autoencoder is used. The contourplot is colored red whenever the MSE is below a set threshold $\epsilon < 0.1$ to indicate a near-perfect reconstruction. Note that the color scaling is exponential to better visualize the MSE loss.
• Figure 2: Plots of the contours of the reconstruction loss in the 2D latent space of a convolutional autoencoder when applied on subset of MNIST (a, b), plots of constructed adversarial anomalies (c, d), and a plot of non-problematic out-of-bounds reconstruction (e). Subplots (a, c) show the results for an autoencoder trained on digits $4$, $5$, and $7$, and Subplots (b, d, e) show the results for an autoencoder trained on digits $0$, and $1$. The visualized samples, i.e. the points in (a, b) are the latent representations of the training data. The shown digits are constructed by sampling from the ${\epsilon} < 0.1$ zone within the marked area, and correspond to these from left to right. The contourplot is colored red whenever the MSE is below a set threshold $\epsilon < 0.1$ to indicate a near-perfect reconstruction. Note that the color scaling is exponential to better visualize the MSE loss.