Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards a Trustworthy Anomaly Detection for Critical Applications through Approximated Partial AUC Loss

Arnaud Bougaham, Benoît Frénay

TL;DR

This work addresses trustworthy anomaly detection in critical domains by enforcing zero false negatives under a limited false positive rate. It introduces tapAUC, an adaptive partial AUC loss guiding the classifier to achieve high true positive rates while keeping false positives low, with the ZFN threshold kept for test-time evaluation. Evaluations on six datasets spanning industrial, medical, and cybersecurity tasks show a mean TPR of 92.52% at a 20.43% FPR, outperforming related partial AUC methods in the targeted region and enabling an uncertainty interval for human-in-the-loop checks. The approach advances practical trustworthiness for critical anomaly detection systems and suggests avenues for smoother training and end-to-end image integration.

Abstract

Anomaly Detection is a crucial step for critical applications such in the industrial, medical or cybersecurity domains. These sectors share the same requirement of handling differently the different types of classification errors. Indeed, even if false positives are acceptable, false negatives are not, because it would reflect a missed detection of a quality issue, a disease or a cyber threat. To fulfill this requirement, we propose a method that dynamically applies a trustworthy approximated partial AUC ROC loss (tapAUC). A binary classifier is trained to optimize the specific range of the AUC ROC curve that prevents the True Positive Rate (TPR) to reach 100% while minimizing the False Positive Rate (FPR). The optimal threshold that does not trigger any false negative is then kept and used at the test step. The results show a TPR of 92.52% at a 20.43% FPR for an average across 6 datasets, representing a TPR improvement of 4.3% for a FPR cost of 12.2% against other state-of-the-art methods. The code is available at https://github.com/ArnaudBougaham/tapAUC.

Towards a Trustworthy Anomaly Detection for Critical Applications through Approximated Partial AUC Loss

TL;DR

This work addresses trustworthy anomaly detection in critical domains by enforcing zero false negatives under a limited false positive rate. It introduces tapAUC, an adaptive partial AUC loss guiding the classifier to achieve high true positive rates while keeping false positives low, with the ZFN threshold kept for test-time evaluation. Evaluations on six datasets spanning industrial, medical, and cybersecurity tasks show a mean TPR of 92.52% at a 20.43% FPR, outperforming related partial AUC methods in the targeted region and enabling an uncertainty interval for human-in-the-loop checks. The approach advances practical trustworthiness for critical anomaly detection systems and suggests avenues for smoother training and end-to-end image integration.

Abstract

Anomaly Detection is a crucial step for critical applications such in the industrial, medical or cybersecurity domains. These sectors share the same requirement of handling differently the different types of classification errors. Indeed, even if false positives are acceptable, false negatives are not, because it would reflect a missed detection of a quality issue, a disease or a cyber threat. To fulfill this requirement, we propose a method that dynamically applies a trustworthy approximated partial AUC ROC loss (tapAUC). A binary classifier is trained to optimize the specific range of the AUC ROC curve that prevents the True Positive Rate (TPR) to reach 100% while minimizing the False Positive Rate (FPR). The optimal threshold that does not trigger any false negative is then kept and used at the test step. The results show a TPR of 92.52% at a 20.43% FPR for an average across 6 datasets, representing a TPR improvement of 4.3% for a FPR cost of 12.2% against other state-of-the-art methods. The code is available at https://github.com/ArnaudBougaham/tapAUC.

Paper Structure

This paper contains 13 sections, 8 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. The tapAUC Method
  4. Zero False Negatives under Limited False Positive Rate
  5. Partial AUC ROC Curve
  6. Loss Function
  7. Algorithm
  8. Evaluation
  9. Datasets
  10. Experimental Setup
  11. Results
  12. Discussion
  13. Conclusion

Figures (4)

  • Figure 1: Anomaly score data points and distribution example of negative (in green) and positive (in red) instances. Unlike the threshold that yields the maximum accuracy (STD Threshold dashed black vertical line), the one that reaches 100% TPR (ZFN Threshold dashed red vertical line) is highly influenced by the positive and the most challenging negative data. Lower score for these challenging negatives and higher score for positives would end up with a full TPR classifier, with limited FPR.
  • Figure 2: Confusion Matrix that maximizes the accuracy.
  • Figure 4: During the classifier training, epoch after epoch, the negative instances that just prevent a 100% TPR and all the positive ones are selected. The customized loss function approximates the partial AUC ROC curve, where the focused improved region helps minimizing the FPR while maintaining the TPR at 100%.
  • Figure 5: Three examples of normal (left green-framed part) and abnormal images (right red-framed part) for the all the datasets coming from image databases. One can observe the very slight difference between the two classes.