Unsupervised anomaly detection algorithms on real-world data: how many do we need?

TL;DR

This large-scale, real-world benchmark compares 32 unsupervised anomaly detection algorithms across 52 multivariate datasets, revealing a robust separation between local and global anomaly problems. The study finds that the $k$-thNN method often dominates overall performance, while EIF excels on global anomalies and IF remains a strong, efficient baseline. Together, these findings support a practical toolbox of three algorithms—$k$-thNN, $k$NN, and EIF—for broad real-world coverage, and highlight the importance of dataset characteristics in algorithm selection. All code and data are openly accessible to facilitate reproducibility and extension.

Abstract

In this study we evaluate 32 unsupervised anomaly detection algorithms on 52 real-world multivariate tabular datasets, performing the largest comparison of unsupervised anomaly detection algorithms to date. On this collection of datasets, the $k$-thNN (distance to the $k$-nearest neighbor) algorithm significantly outperforms the most other algorithms. Visualizing and then clustering the relative performance of the considered algorithms on all datasets, we identify two clear clusters: one with ``local'' datasets, and another with ``global'' datasets. ``Local'' anomalies occupy a region with low density when compared to nearby samples, while ``global'' occupy an overall low density region in the feature space. On the local datasets the $k$NN ($k$-nearest neighbor) algorithm comes out on top. On the global datasets, the EIF (extended isolation forest) algorithm performs the best. Also taking into consideration the algorithms' computational complexity, a toolbox with these three unsupervised anomaly detection algorithms suffices for finding anomalies in this representative collection of multivariate datasets. By providing access to code and datasets, our study can be easily reproduced and extended with more algorithms and/or datasets.

Figures (5)

• Figure 1: 8 examples of different types of anomalies along the 4 defined property axes. Normal data are visualized as blue points, while anomalies are visualized as red crosses.
• Figure 2: Boxplots of the performance of each algorithm on each dataset in terms of percentage of maximum AUC. The maximum AUC is the highest AUC value obtained by the best performing algorithm on that particular dataset. The whiskers in the boxplots extend 1.5 times the interquartile range past the low and high quartiles. Dataset-algorithm combinations outside of the whiskers are marked as diamonds.
• Figure 3: Clustered heatmap of the ROC/AUC performance of each algorithm. The algorithms and datasets are each clustered using hierarchical clustering with average linkage and the Pearson correlation as metric.
• Figure 4: Boxplots of the performance of each algorithm on the "local" datasets in terms of percentage of maximum AUC. The maximum AUC is the highest AUC value obtained by the best performing algorithm on that particular dataset. The whiskers in the boxplots extend 1.5 times the interquartile range past the low and high quartiles. Dataset-algorithm combinations outside of the whiskers are marked as diamonds.
• Figure 5: Boxplots of the performance of each algorithm on the global datasets in terms of percentage of maximum AUC. The maximum AUC is the highest AUC value obtained by the best performing algorithm on that particular dataset. The whiskers in the boxplots extend 1.5 times the interquartile range past the low and high quartiles. Dataset-algorithm combinations outside of the whiskers are marked as diamonds.