Quantifying Explanation Quality in Graph Neural Networks using Out-of-Distribution Generalization

TL;DR

This work tackles the challenge of evaluating GNN explanations by focusing on causal relevance under distribution shifts. It introduces the Explanation Generalization Score (EGS) and trains Explanation-Guided GNNs (EG-GNNs) using representative explanations to assess how well explanations align with invariant causal subgraphs. Through large-scale experiments on synthetic motifs (Triangle, Pentagon) and real-world molecular datasets (Fc, Mutag), the study shows that ground-truth explanations yield positive EGS across OOD splits, while explainers approximate this signal with varying fidelity. The proposed OOD-centered framework decouples explainer assessment from baseline performance, offering a robust benchmark with practical implications for incorporating domain knowledge into robust graph learning and explainability evaluation.

Abstract

Evaluating the quality of post-hoc explanations for Graph Neural Networks (GNNs) remains a significant challenge. While recent years have seen an increasing development of explainability methods, current evaluation metrics (e.g., fidelity, sparsity) often fail to assess whether an explanation identifies the true underlying causal variables. To address this, we propose the Explanation-Generalization Score (EGS), a metric that quantifies the causal relevance of GNN explanations. EGS is founded on the principle of feature invariance and posits that if an explanation captures true causal drivers, it should lead to stable predictions across distribution shifts. To quantify this, we introduce a framework that trains GNNs using explanatory subgraphs and evaluates their performance in Out-of-Distribution (OOD) settings (here, OOD generalization serves as a rigorous proxy for the explanation's causal validity). Through large-scale validation involving 11,200 model combinations across synthetic and real-world datasets, our results demonstrate that EGS provides a principled benchmark for ranking explainers based on their ability to capture causal substructures, offering a robust alternative to traditional fidelity-based metrics.

Figures (24)

• Figure 2: SCM of data-generating process
• Figure 3: Egs comparison for ground truth and graph explainers.Egs results using ground-truth explanations and explanations generated from eight graph explainers on Triangle (left) and Mutag (right). We evaluate the results over NumSubgraphs and MolWt properties for Triangle and Mutag, respectively. GT indicates Ground Truth Explanations. We use GAT as the backbone models. More results in Appendix \ref{['app:figure']}.
• Figure 4: Egs results using ground-truth explanations on all GNN backbones.Egs results across four OOD splits on Triangle, Pentagon, Fc, and Mutag using all GNN backbones: GAT, GCN, SAGE, and GIN. We evaluate the results over NumSubgraphs and MolWt for Triangle and Mutag, respectively.
• Figure 5: Egs results using ground-truth explanations on all dataset-property pairs.Egs results across four OOD splits on Triangle, Pentagon, Fc, and Mutag, using GAT model. For each dataset, we evaluate all of its corresponding graph or molecular properties.
• Figure 6: Faithfulness of ground-truth explanations for EG-GNNs across four datasets. Distribution of faithfulness scores computed using ground-truth explanations for EG-GNNs on Triangle, Pentagon, Fc, and Mutag. Each box summarizes faithfulness over all test samples for all OOD splits, properties, and GNN backbones. The central line in each box denotes the median; boxes indicate the interquartile range (IQR); whiskers extend to 1.5$\times$IQR; and outlier points are omitted for readability.