Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PROVEX: Enhancing SOC Analyst Trust with Explainable Provenance-Based IDS

Devang Dhanuka, Nidhi Rastogi

TL;DR

This work tackles the trust gap in provenance-based intrusion detection by integrating an explainability layer into the TGNN-based KAIROS IDS. The PROVEX framework combines GraphMask, GNNExplainer, and a novel VA-TGExplainer to produce window-level and per-event explanations that quantify uncertainty. Evaluated on the CADETS DARPA dataset, PROVEX demonstrates faithful explanations with measurable fidelity (comprehensiveness 0.82, sufficiency 0.15) and practical overheads (3-5 seconds per event), aiding analyst triage. While focusing on system integration rather than new algorithms, the approach provides a deployable path toward trustworthy, interpretable graph-based cyber defense in SOC environments.

Abstract

Modern intrusion detection systems (IDS) leverage graph neural networks (GNNs) to detect malicious activity in system provenance data, but their decisions often remain a black box to analysts. This paper presents a comprehensive XAI framework designed to bridge the trust gap in Security Operations Centers (SOCs) by making graph-based detection transparent. We implement this framework on top of KAIROS, a state-of-the-art temporal graph-based IDS, though our design is applicable to any temporal graph-based detector with minimal adaptation. The complete codebase is available at https://github.com/devang1304/provex.git. We augment the detection pipeline with post-hoc explanations that highlight why an alert was triggered, identifying key causal subgraphs and events. We adapt three GNN explanation methods - GraphMask, GNNExplainer, and a variational temporal GNN explainer (VA-TGExplainer) - to the temporal provenance context. These tools output human-interpretable representations of anomalous behavior, including important edges and uncertainty estimates. Our contributions focus on the practical integration of these explainers, addressing challenges in memory management and reproducibility. We demonstrate our framework on the DARPA CADETS Engagement 3 dataset and show that it produces concise window-level explanations for detected attacks. Our evaluation reveals that the explainers preserve the TGNN's decisions with high fidelity, surfacing critical edges such as malicious file interactions and anomalous netflows. The average explanation overhead is 3-5 seconds per event. By providing insight into the model's reasoning, our framework aims to improve analyst trust and triage speed.

PROVEX: Enhancing SOC Analyst Trust with Explainable Provenance-Based IDS

TL;DR

This work tackles the trust gap in provenance-based intrusion detection by integrating an explainability layer into the TGNN-based KAIROS IDS. The PROVEX framework combines GraphMask, GNNExplainer, and a novel VA-TGExplainer to produce window-level and per-event explanations that quantify uncertainty. Evaluated on the CADETS DARPA dataset, PROVEX demonstrates faithful explanations with measurable fidelity (comprehensiveness 0.82, sufficiency 0.15) and practical overheads (3-5 seconds per event), aiding analyst triage. While focusing on system integration rather than new algorithms, the approach provides a deployable path toward trustworthy, interpretable graph-based cyber defense in SOC environments.

Abstract

Modern intrusion detection systems (IDS) leverage graph neural networks (GNNs) to detect malicious activity in system provenance data, but their decisions often remain a black box to analysts. This paper presents a comprehensive XAI framework designed to bridge the trust gap in Security Operations Centers (SOCs) by making graph-based detection transparent. We implement this framework on top of KAIROS, a state-of-the-art temporal graph-based IDS, though our design is applicable to any temporal graph-based detector with minimal adaptation. The complete codebase is available at https://github.com/devang1304/provex.git. We augment the detection pipeline with post-hoc explanations that highlight why an alert was triggered, identifying key causal subgraphs and events. We adapt three GNN explanation methods - GraphMask, GNNExplainer, and a variational temporal GNN explainer (VA-TGExplainer) - to the temporal provenance context. These tools output human-interpretable representations of anomalous behavior, including important edges and uncertainty estimates. Our contributions focus on the practical integration of these explainers, addressing challenges in memory management and reproducibility. We demonstrate our framework on the DARPA CADETS Engagement 3 dataset and show that it produces concise window-level explanations for detected attacks. Our evaluation reveals that the explainers preserve the TGNN's decisions with high fidelity, surfacing critical edges such as malicious file interactions and anomalous netflows. The average explanation overhead is 3-5 seconds per event. By providing insight into the model's reasoning, our framework aims to improve analyst trust and triage speed.

Paper Structure

This paper contains 39 sections, 2 equations, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Intrusion Detection on Provenance Graphs
  4. Static GNN Explainability
  5. Temporal and Dynamic Graph Explainability
  6. System Overview
  7. Event Log Ingestion
  8. TGNN Encoder-Decoder (KAIROS Core)
  9. Anomaly Detection & Alerting
  10. Attack Reconstruction
  11. XAI Module (Explanation)
  12. SOC Dashboard (Analyst Interface)
  13. Methodology
  14. Explanation Pipeline
  15. Attack Window Identification
  16. ...and 24 more sections

Figures (3)

  • Figure 1: PROVEX domain model. Input artifacts flow through the TGN model and EventContext to a TemporalLinkWrapper, which feeds the explanation pipeline. Three explainers (GraphMask, VA-TGExplainer, GNNExplainer) generate edge importance masks that are aggregated into JSON reports and dashboard visualizations.
  • Figure 2: KAIROS TGNN Encoder-Decoder architecture.
  • Figure 3: Sample JSON output from the explanation pipeline.