AnomalyAID: Reliable Interpretation for Semi-supervised Network Anomaly Detection

TL;DR

AnomalyAID addresses the interpretability gap in semi-supervised network anomaly detection by introducing a dual-component framework. The Global-local Knowledge Association Mechanism (KAM) provides reliable explanations by aligning local and cluster-level interpretations, while the Two-stage Semi-supervised Learning (ToS) enables high-confidence pseudo-labeling through a two-round training process. Evaluations on three widely used network anomaly datasets show that ToS improves detection performance and SPAUC, and that KAM delivers superior fidelity, stability, and robustness compared with existing interpreters. The combined approach offers practical value for security operators by delivering accurate anomaly detection with trustworthy explanations in semi-supervised settings.

Abstract

Semi-supervised Learning plays a crucial role in network anomaly detection applications, however, learning anomaly patterns with limited labeled samples is not easy. Additionally, the lack of interpretability creates key barriers to the adoption of semi-supervised frameworks in practice. Most existing interpretation methods are developed for supervised/unsupervised frameworks or non-security domains and fail to provide reliable interpretations. In this paper, we propose AnomalyAID, a general framework aiming to (1) make the anomaly detection process interpretable and improve the reliability of interpretation results, and (2) assign high-confidence pseudo labels to unlabeled samples for improving the performance of anomaly detection systems with limited supervised data. For (1), we propose a novel interpretation approach that leverages global and local interpreters to provide reliable explanations, while for (2), we design a new two-stage semi-supervised learning framework for network anomaly detection by aligning both stages' model predictions with special constraints. We apply AnomalyAID over two representative network anomaly detection tasks and extensively evaluate AnomalyAID with representative prior works. Experimental results demonstrate that AnomalyAID can provide accurate detection results with reliable interpretations for semi-supervised network anomaly detection systems.

Figures (9)

• Figure 1: Illustration of the divergent explanation results of different interpreters.
• Figure 2: Six example explanation results for the same input data with the same model using LIME, DeepLift, SHAPE lundberg2017unified_shap, COIN liu2017contextual_coin, CADE yang2021cade, and xNIDS wei2023xnids. Different interpreters give different explanations.
• Figure 3: Overview of the proposed AnomalyAID.
• Figure 4: Fidelity, stability, and robustness evaluation of interpreters on ISCXTor2016.
• Figure 5: Fidelity, stability, and robustness evaluation of interpreters on CIC-DoHBrw-2020.