Evaluating Explanation Quality in X-IDS Using Feature Alignment Metrics

TL;DR

This work addresses the gap in evaluating explanations from X-IDS by introducing three domain-alignment metrics—Feature Alignment Precision ($\text{FAP}$), Feature Alignment Recall ($\text{FAR}$), and Feature Alignment F1 ($\text{FAF1}$)—that compare top-$k$ model explanations to predefined domain-informed feature sets derived from MITRE ATT&CK and D3FEND. The authors formalize the system model, define the metrics at instance, class, and dataset levels, and apply them to SHAP explanations from three X-IDS models (Random Forest, DNN, CNN-BiLSTM) trained on CICIDS2017. Experimental results show that deep learning models yield explanations more aligned with domain knowledge than RF, with higher FAF1 at lower $k$ values, suggesting more actionable insights for security analysts. The study provides a practical framework for tailoring explanations to domain knowledge, aiding analyst triage and guiding future improvements in domain-informed feature sets and evaluation practices.

Abstract

Explainable artificial intelligence (XAI) methods have become increasingly important in the context of explainable intrusion detection systems (X-IDSs) for improving the interpretability and trustworthiness of X-IDSs. However, existing evaluation approaches for XAI focus on model-specific properties such as fidelity and simplicity, and neglect whether the explanation content is meaningful or useful within the application domain. In this paper, we introduce new evaluation metrics measuring the quality of explanations from X-IDSs. The metrics aim at quantifying how well explanations are aligned with predefined feature sets that can be identified from domain-specific knowledge bases. Such alignment with these knowledge bases enables explanations to reflect domain knowledge and enables meaningful and actionable insights for security analysts. In our evaluation, we demonstrate the use of the proposed metrics to evaluate the quality of explanations from X-IDSs. The experimental results show that the proposed metrics can offer meaningful differences in explanation quality across X-IDSs and attack types, and assess how well X-IDS explanations reflect known domain knowledge. The findings of the proposed metrics provide actionable insights for security analysts to improve the interpretability of X-IDS in practical settings.

Figures (4)

• Figure 1: High-level overview of the explanation evaluation process. The top‑$k$ features from the XAI model are evaluated against domain-informed feature sets.
• Figure 2: (a) FAR and (b) FAP at various top-$k$ cutoffs for DNN, RF, and CNN-BiLSTM X-IDSs. These metrics show how well each X-IDS's top-$k$ features align with the set of domain-informed features across the entire dataset.
• Figure 3: Class level explanation evaluation metrics across attack types for the DNN model. (a) FAR trends showing how many of the domain-informed features are captured as $k$ increases. (b) FAP trends showing how many of domain-informed features are captured by the explanations at each $k$.
• Figure 4: (a) Class Level FAF1 curve illustrating how each model’s top-$k$ features balance correctness (FAP) and completeness (FAR) with respect to the MITRE-based features. (b) Class Level Feature alignment Precision–Recall (FPR) curve, showing how FAP varies with FAR for different $k$ values.