Unsupervised Outlier Detection using Random Subspace and Subsampling Ensembles of Dirichlet Process Mixtures

TL;DR

This work tackles unsupervised outlier detection by marrying Dirichlet process Gaussian mixtures with variational inference and two ensemble strategies: random subspace projections and instance subsampling. The resulting OEDPM method automatically determines the number of mixture components, mitigates outlier-induced bias, and remains scalable to large, high-dimensional data through efficient VI and ensemble aggregation of likelihoods. Empirical results on 27 ODDS datasets show competitive and often superior F1-scores against a wide range of baselines, with favorable runtimes. The approach yields an interpretable, probabilistic detector suitable for non-Gaussian features, and opens avenues for extending subspace projections to nonlinear regimes.

Abstract

Probabilistic mixture models are recognized as effective tools for unsupervised outlier detection owing to their interpretability and global characteristics. Among these, Dirichlet process mixture models stand out as a strong alternative to conventional finite mixture models for both clustering and outlier detection tasks. Unlike finite mixture models, Dirichlet process mixtures are infinite mixture models that automatically determine the number of mixture components based on the data. Despite their advantages, the adoption of Dirichlet process mixture models for unsupervised outlier detection has been limited by challenges related to computational inefficiency and sensitivity to outliers in the construction of outlier detectors. Additionally, Dirichlet process Gaussian mixtures struggle to effectively model non-Gaussian data with discrete or binary features. To address these challenges, we propose a novel outlier detection method that utilizes ensembles of Dirichlet process Gaussian mixtures. This unsupervised algorithm employs random subspace and subsampling ensembles to ensure efficient computation and improve the robustness of the outlier detector. The ensemble approach further improves the suitability of the proposed method for detecting outliers in non-Gaussian data. Furthermore, our method uses variational inference for Dirichlet process mixtures, which ensures both efficient and rapid computation. Empirical analyses using benchmark datasets demonstrate that our method outperforms existing approaches in unsupervised outlier detection.

Figures (6)

• Figure 1: Random projections of two-dimensional data onto one dimension. In the upper panels, outlier A becomes evident through the first random projection (middle), while outliers B and C are captured using the second random projection axis (right). In the lower panels, the original two-dimensional data exhibit a significant bias when modeled with Gaussian mixtures (left). The one-dimensional projected datasets are shown to be more suitable for Gaussian mixture modeling.
• Figure 2: Two-dimensional random projections of ten-dimensional non-Gaussian data of size $1000$. The left panel shows a random projection of data generated uniformly on the unit cube $[0,1]^{10}$, i.e., $\prod_{j=1}^{10}\text{Uniform}(0,1)$. The right panel shows a random projection of data generated uniformly on the vertices of the unit cube $[0,1]^{10}$, i.e., $\prod_{j=1}^{10}\text{Bernoulli}(1/2)$.
• Figure 3: Log-densities estimated by DPGM for the original data (left) and the subsampled data (right). The patterns in the original data are well captured by the subsampled data.
• Figure 4: Graphical illustration of OEDPM. The three-dimensional original dataset is fitted using variational DPGM with random projection and subsampling (see Section \ref{['VR']}). Mixture components with small weights are excluded to minimize the influence of outliers when constructing outlier detectors (see Section \ref{['inlier selection']}). After pruning the mixture components, outlier scores are calculated based on the likelihood (see Section \ref{['outlier score']}).
• Figure 5: Sensitivity analysis for the contamination parameter $\phi$ using the 27 benchmark datasets.