Mitigating Long-Tailed Anomaly Score Distributions with Importance-Weighted Loss

TL;DR

The paper addresses LTD in anomaly-score distributions caused by diverse normal data, which biases standard detectors toward the majority. It introduces the Importance-Weighted Loss (IWL), which uses a Box-Cox transformation to induce a Gaussian-like anomaly-score distribution and importance sampling to reweight samples without requiring prior normal-class knowledge. The approach includes skewness-based weight bounding, outlier-aware Gaussian estimation, and a principled loss to train with reweighted samples. Extensive experiments on three image datasets and three hyperspectral datasets show consistent improvements (e.g., an average AUROC gain of $0.043$ over baselines), supporting IWL's practicality for real-world LTD mitigation in anomaly detection. The work offers a simple, architecture-agnostic modification to the loss function that enhances robustness to long-tailed normal data patterns in industrial contexts.

Abstract

Anomaly detection is crucial in industrial applications for identifying rare and unseen patterns to ensure system reliability. Traditional models, trained on a single class of normal data, struggle with real-world distributions where normal data exhibit diverse patterns, leading to class imbalance and long-tailed anomaly score distributions (LTD). This imbalance skews model training and degrades detection performance, especially for minority instances. To address this issue, we propose a novel importance-weighted loss designed specifically for anomaly detection. Compared to the previous method for LTD in classification, our method does not require prior knowledge of normal data classes. Instead, we introduce a weighted loss function that incorporates importance sampling to align the distribution of anomaly scores with a target Gaussian, ensuring a balanced representation of normal data. Extensive experiments on three benchmark image datasets and three real-world hyperspectral imaging datasets demonstrate the robustness of our approach in mitigating LTD-induced bias. Our method improves anomaly detection performance by 0.043, highlighting its effectiveness in real-world applications.

Figures (5)

• Figure 1: Analysis of the hyperspectral anomaly detection dataset 963e-1d34-24. We analyze the hyperspectral anomaly detection dataset and visualize the data using t-SNE hinton2002stochastic (left). The distribution of anomaly scores (right) exhibits a long-tailed pattern, highlighting the inherent class imbalance and its impact on anomaly detection.
• Figure 2: Distribution of log anomaly scores in the MNIST dataset. We designate classes "6" and "7" as the majority and minority classes, respectively. To construct a balanced dataset, both classes are assigned an equal number of samples. Conversely, in the imbalanced dataset, the minority class comprises only one-tenth of the samples in the majority class. The introduction of class imbalance leads to an increase in skewness from 0.504 to 0.524, indicating that class imbalance inherently induces a long-tailed distribution in anomaly score distributions.
• Figure 3: Application of the Box-Cox transformation to normalize a skewed distribution, resulting in a Gaussian-like distribution.
• Figure 4: Visualize the distribution of log-scaled scores on the MNIST training dataset using DSVDD.
• Figure 5: Evaluation across various $\beta$ value with three image datasets.