Anomaly Detection with Adaptive and Aggressive Rejection for Contaminated Training Data

TL;DR

The paper tackles anomaly detection when training data are contaminated, a situation where fixed contamination ratios degrade performance. It introduces Adaptive and Aggressive Rejection (AAR), which combines a mini-batch based modified z-score hard rejection with a GMM-based soft rejection to dynamically identify and down-weight anomalies while preserving normal samples. The approach is supported by theoretical insights showing why aggressive rejection helps under distributional overlap and by extensive experiments on image (MNIST, F-MNIST) and 30 tabular datasets, achieving a notable AUROC gain of about $0.041$ over state-of-the-art methods. The method is scalable and robust to varying contamination levels, with practical implications for security, healthcare, and other real-world domains.

Abstract

Handling contaminated data poses a critical challenge in anomaly detection, as traditional models assume training on purely normal data. Conventional methods mitigate contamination by relying on fixed contamination ratios, but discrepancies between assumed and actual ratios can severely degrade performance, especially in noisy environments where normal and abnormal data distributions overlap. To address these limitations, we propose Adaptive and Aggressive Rejection (AAR), a novel method that dynamically excludes anomalies using a modified z-score and Gaussian mixture model-based thresholds. AAR effectively balances the trade-off between preserving normal data and excluding anomalies by integrating hard and soft rejection strategies. Extensive experiments on two image datasets and thirty tabular datasets demonstrate that AAR outperforms the state-of-the-art method by 0.041 AUROC. By providing a scalable and reliable solution, AAR enhances robustness against contaminated datasets, paving the way for broader real-world applications in domains such as security and healthcare.

Figures (5)

• Figure 1: Motivation for aggressive rejection. (a) demonstrates that a 5–10% higher rejection ratio $\gamma$ yields better performance than using the true contamination ratio ($\gamma=\gamma_0$). (b) illustrates the distribution of anomaly scores of normal and abnormal samples. (c) presents the robustness across quantiles. Notably, while a 0.9-quantile rejection achieves a robustness of 0.63 and an AUROC of 0.928, a more aggressive 0.8-quantile rejection outperforms it with a higher robustness of 0.69 and an AUROC of 0.936.
• Figure 2: Thresholds applied to two Gaussian models with contamination ratios of 20% (left) and 1% (right) demonstrate significant variations in performance. For datasets with a high contamination ratio of 20%, the intersection threshold which is defined as the point where normal and abnormal distributions overlap effectively excludes anomalies, thereby optimizing robustness. Conversely, the MZ threshold proves inadequate, failing to sufficiently exclude all anomalies. However, on datasets with a low contamination ratio of 1%, the intersection threshold excludes numerous normal data points, adversely affecting stability. In such cases, the $z$-$\sigma$ threshold (e.g., $z=3$) offers a balanced solution, achieving superior robustness compared to the MZ threshold while providing greater stability than the intersection threshold. This balance makes the $z$-$\sigma$ threshold a more reliable and versatile choice for handling datasets with varying levels of contamination.
• Figure 3: Validation of aggressive rejection with various rejection ratio on MNIST dataset.
• Figure 4: Evaluation across various contamination ratios by training the AE model on MNIST, F-MNIST, arrhythmia, and thyroid datasets, incorporating representative robust methods for comparison.
• Figure 5: Sensitivity of $z$ (left) and $t_s$ (right) on MNIST dataset with AE. Bold, colored dashed line, and grey dashed line indicates AUROC using $\tau_{\sigma}$ which depends on $z$ value, intersection threshold $\tau_{I}$, and only MZ threshold, respectively.

Theorems & Definitions (3)

• Definition 3.1: Robustness of a rejection threshold
• Proposition 3.1
• proof