Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Weakly-supervised anomaly detection for multimodal data distributions

Xu Tan, Junqi Chen, Sylwan Rahardja, Jiawei Yang, Susanto Rahardja

TL;DR

This work tackles weakly-supervised anomaly detection in multimodal data by introducing WVAD, a two-component framework that combines a deep variational mixture model with an anomaly score estimator. The variational model partitions data into latent clusters using $p(y)=Cat(\pi)$ and $p(\mathbf{z}|y)=\mathcal{N}(\mathbf{z}|\mu_{\mathbf{z},y},\sigma_{\mathbf{z},y}^2)$ and learns via a factorized posterior $q_\phi(y,\mathbf{z}|\mathbf{x})$, optimizing ELBO terms for unlabeled data and labeled anomalies; from this, five features $[\mathbf{y}; \mathbf{z}; f_e; f_r; f_c]$ are produced and passed to a sigmoid-ended anomaly score estimator. The training strategy uses a two-stage approach: pretraining the autoencoder-like mixture model with $L_1$ to initialize a Gaussian mixture via EM, followed by joint training with $L_2 = -\mathcal{L}_{ELBO} + \lambda L_{CE}$ and a scheduled\ $\lambda$ to balance representation learning and scoring. Evaluation on Letter, Ionosphere, and Satellite demonstrates WVAD’s superiority over six state-of-the-art baselines under 10% and 5% labeling, underscoring its effectiveness for multimodal weakly-supervised anomaly detection. The findings suggest WVAD’s practical impact in real-world settings where labeled anomalies are scarce but data originate from multiple clusters with distinct characteristics.

Abstract

Weakly-supervised anomaly detection can outperform existing unsupervised methods with the assistance of a very small number of labeled anomalies, which attracts increasing attention from researchers. However, existing weakly-supervised anomaly detection methods are limited as these methods do not factor in the multimodel nature of the real-world data distribution. To mitigate this, we propose the Weakly-supervised Variational-mixture-model-based Anomaly Detector (WVAD). WVAD excels in multimodal datasets. It consists of two components: a deep variational mixture model, and an anomaly score estimator. The deep variational mixture model captures various features of the data from different clusters, then these features are delivered to the anomaly score estimator to assess the anomaly levels. Experimental results on three real-world datasets demonstrate WVAD's superiority.

Weakly-supervised anomaly detection for multimodal data distributions

TL;DR

This work tackles weakly-supervised anomaly detection in multimodal data by introducing WVAD, a two-component framework that combines a deep variational mixture model with an anomaly score estimator. The variational model partitions data into latent clusters using and and learns via a factorized posterior , optimizing ELBO terms for unlabeled data and labeled anomalies; from this, five features are produced and passed to a sigmoid-ended anomaly score estimator. The training strategy uses a two-stage approach: pretraining the autoencoder-like mixture model with to initialize a Gaussian mixture via EM, followed by joint training with and a scheduled\ to balance representation learning and scoring. Evaluation on Letter, Ionosphere, and Satellite demonstrates WVAD’s superiority over six state-of-the-art baselines under 10% and 5% labeling, underscoring its effectiveness for multimodal weakly-supervised anomaly detection. The findings suggest WVAD’s practical impact in real-world settings where labeled anomalies are scarce but data originate from multiple clusters with distinct characteristics.

Abstract

Weakly-supervised anomaly detection can outperform existing unsupervised methods with the assistance of a very small number of labeled anomalies, which attracts increasing attention from researchers. However, existing weakly-supervised anomaly detection methods are limited as these methods do not factor in the multimodel nature of the real-world data distribution. To mitigate this, we propose the Weakly-supervised Variational-mixture-model-based Anomaly Detector (WVAD). WVAD excels in multimodal datasets. It consists of two components: a deep variational mixture model, and an anomaly score estimator. The deep variational mixture model captures various features of the data from different clusters, then these features are delivered to the anomaly score estimator to assess the anomaly levels. Experimental results on three real-world datasets demonstrate WVAD's superiority.
Paper Structure (14 sections, 7 equations, 3 figures, 2 tables)

This paper contains 14 sections, 7 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Problem statement
  5. Model architecture
  6. Training strategy
  7. Experiments
  8. Datasets
  9. Letter
  10. Ionosphere
  11. Satellite
  12. Baselines and settings
  13. Experimental results
  14. Conclusions

Figures (3)

  • Figure 1: An overview of the whole model architecture of WVAD.
  • Figure 2: The 2-D visualizations of the Letter, Ionosphere, and Satellite datasets. The normal data are marked in blue, while the anomalies are marked as red. One can observe three obvious data clusters in Letter, two clusters in Ionosphere, and three clusters in Satellite.
  • Figure 3: The 2-D visualizations of the latent features $\mathbf{z}$ extracted from the WVAD, colored by the estimated anomaly scores. The warmer the color, the higher the anomaly score. The true anomalies are surrounded by red circles.