FlowX: Towards Explainable Graph Neural Networks via Message Flows

TL;DR

FlowX presents a novel, flow-centric framework for explaining Graph Neural Networks by attributing predictions to multi-hop message flows. It combines Shapley-value-inspired flow marginal contributions with a fair, permutation-based sampling scheme and a trainable refinement that enables both necessary and sufficient explanations, as well as a FlowMask variant for mutual-information driven explanations. The approach models GNNs as flow-based systems and converts flow attributions into layer-edge masks via stochastic normalization, enabling targeted explanations with quantified fidelity and sparsity. Empirical results across synthetic and real-world datasets show FlowX and its variants outperform edge- or node-centric baselines in both faithfulness (Fidelity+) and sufficiency (Fidelity-), while Flow-based explanations better capture multi-hop dependencies than traditional methods. The work highlights multi-hop correlation modeling as a core advantage of flow-based explanations and provides a scalable, flexible tool for producing human-interpretable GNN explanations with practical impact in domains requiring trustworthy AI.

Abstract

We investigate the explainability of graph neural networks (GNNs) as a step toward elucidating their working mechanisms. While most current methods focus on explaining graph nodes, edges, or features, we argue that, as the inherent functional mechanism of GNNs, message flows are more natural for performing explainability. To this end, we propose a novel method here, known as FlowX, to explain GNNs by identifying important message flows. To quantify the importance of flows, we propose to follow the philosophy of Shapley values from cooperative game theory. To tackle the complexity of computing all coalitions' marginal contributions, we propose a flow sampling scheme to compute Shapley value approximations as initial assessments of further training. We then propose an information-controlled learning algorithm to train flow scores toward diverse explanation targets: necessary or sufficient explanations. Experimental studies on both synthetic and real-world datasets demonstrate that our proposed FlowX and its variants lead to improved explainability of GNNs. The code is available at https://github.com/divelab/DIG.

Figures (10)

• Figure 1: Edge-based explanation challenge. The graph in the figure describes a traffic graph where each node indicates an intersection; each edge denotes a road. The task is to explain why there is a traffic jam at the crossroad and how to construct overpasses to address the problem, where we denote the congested intersection as a red node. The edge-based explanation indicates that the in-flow from $v_1, v_5$ to $v_2$ and the out-flow from $v_2$ to $v_3, v_4$ cause the traffic jam. However, this explanation falls short when it comes to providing the fine-grained traffic flow data necessary for overpass construction planning. In such case, the edge-based explanation is unspecified since it gives rise to two equally valid, but mutually contradictory, flow interpretations that imply conflicting overpass construction strategies, i.e., $v_1\rightarrow v_2 \rightarrow v_3$ or $v_1 \rightarrow v_2 \rightarrow v_4$.
• Figure 2: Graph modeling using SCMs. The figure illustrates the assumptions of data generation processes described by Structural Causal Models (SCMs). The grey nodes are observed and can be intervened by us. The white nodes are unobserved or cannot be intervened/operated directly.
• Figure 3: An illustration of our initial assessments via sampling marginal contributions. For each sampling iteration, we iteratively remove one layer edge until all layer edges are removed. In this example, the removed layer edges are shown in bold and purple lines while the corresponding message flows are shown in arrow lines. In the first step, we remove the layer edge between $v_1$ and $v_2$ from the first GNN layer and compute the marginal contribution. Then four message flows (red) are removed and the contribution scores are averaged and assigned to these four message flows. In the second step, we remove the layer edge $\hat{a}^2_{24}$ and distribute the marginal contribution to the corresponding three flows (green).
• Figure 4: Necessary explanation comparison. We compare Fidelity+ values on 9 datasets with GINs under different Sparsity levels. Our methods are drawn in solid lines while baselines are drawn in dashed. Higher Fidelity+ indicates better performance.
• Figure 5: Sufficient explanation comparison. We compare Fidelity- values on 9 datasets with GINs under different Sparsity levels. Our methods are drawn in solid lines while baselines are drawn in dashed. Lower Fidelity- indicates better performance.