Interpreting GNN-based IDS Detections Using Provenance Graph Structural Features

TL;DR

This work tackles the explainability gap in GNN-based intrusion detection on system provenance graphs by introducing ProvExplainer, a surrogate decision-tree framework that uses security-aware discriminant subgraph patterns and graph-structural features to produce instance-level explanations. By mining discriminative motifs with a weighted scoring function and training a surrogate DT to mimic the GNN, ProvExplainer provides verifiable, human-interpretable justifications aligned with real-world security artifacts, achieving superior Fidelity^+ and precision/recall while reducing analyst workload through actionable explanations. The authors validate ProvExplainer on diverse datasets (DARPA APT, Enterprise APT, Supply-Chain APT, and Fileless Malware) and show strong agreement with GNN decisions (often >97%), with consistently improved explainability metrics and a notable reduction in graph traversal distance for actionability. Overall, ProvExplainer offers a domain-specialized, scalable path to trustworthy explanations for provenance-based IDS, enabling more effective triage and response in security operations.

Abstract

Advanced cyber threats (e.g., Fileless Malware and Advanced Persistent Threat (APT)) have driven the adoption of provenance-based security solutions. These solutions employ Machine Learning (ML) models for behavioral modeling and critical security tasks such as malware and anomaly detection. However, the opacity of ML-based security models limits their broader adoption, as the lack of transparency in their decision-making processes restricts explainability and verifiability. We tailored our solution towards Graph Neural Network (GNN)-based security solutions since recent studies employ GNNs to comprehensively digest system provenance graphs for security-critical tasks. To enhance the explainability of GNN-based security models, we introduce PROVEXPLAINER, a framework offering instance-level security-aware explanations using an interpretable surrogate model. PROVEXPLAINER's interpretable feature space consists of discriminant subgraph patterns and graph structural features, which can be directly mapped to the system provenance problem space, making the explanations human interpretable. We show how PROVEXPLAINER synergizes with current state-of-the-art (SOTA) GNN explainers to deliver domain and instance-specific explanations. We measure the explanation quality using the Fidelity+/Fidelity- metric as used by traditional GNN explanation literature, we incorporate the precision/recall metric, where we consider the accuracy of the explanation against the ground truth, and we designed a human actionability metric based on graph traversal distance. On real-world Fileless and APT datasets, PROVEXPLAINER achieves up to 29%/27%/25%/1.4x higher Fidelity+, precision, recall, and actionability (where higher values are better), and 12% lower Fidelity- (where lower values are better) when compared against SOTA GNN explainers.

Figures (9)

• Figure 1: Trace: after an employee clicks on a phishing link, Firefox installs multiple Trojans to exfiltrate sensitive data.
• Figure 2: ProvExplainer architecture.
• Figure 3: Agreement (higher is better) measured using WMA F1 of surrogate decision trees versus the GAT model across different feature sets.
• Figure 4: Comparison of GAT explainers in identifying important structures using $\text{fidelity}^{+}$ (higher is better) and $\text{fidelity}^{-}$ (lower is better).
• Figure 5: Effectiveness of gnn explainers at identifying documented entities, measured using precision and recall (higher is better for both).