Iterative Graph Neural Network Enhancement via Frequent Subgraph Mining of Explanations

TL;DR

Experimental evidence is presented, with synthetic and real-world data, which show that EEGL outperforms related approaches in predictive performance and that it has a node-distinguishing power beyond that of vanilla GNNs.

Abstract

We formulate an XAI-based model improvement approach for Graph Neural Networks (GNNs) for node classification, called Explanation Enhanced Graph Learning (EEGL). The goal is to improve predictive performance of GNN using explanations. EEGL is an iterative self-improving algorithm, which starts with a learned "vanilla" GNN, and repeatedly uses frequent subgraph mining to find relevant patterns in explanation subgraphs. These patterns are then filtered further to obtain application-dependent features corresponding to the presence of certain subgraphs in the node neighborhoods. Giving an application-dependent algorithm for such a subgraph-based extension of the Weisfeiler-Leman (1-WL) algorithm has previously been posed as an open problem. We present experimental evidence, with synthetic and real-world data, which show that EEGL outperforms related approaches in predictive performance and that it has a node-distinguishing power beyond that of vanilla GNNs. We also analyze EEGL's training dynamics.

Figures (26)

• Figure 1: The motifs used in the dataset generation: The "house" motif (a) and its variant (b), and motif pairs with 1-WL indistinguishable nodes (c,d).
• Figure 2: Example of the confusion matrices for a fold of $G(M_{2}')$.
• Figure 3: The $d=10$ maximal frequent subgraphs extracted by EEGL in the first (R0 $\to$ R1) iteration for a fold of $G(M_{2}')$. Class labels indicated on top. There are two patterns for label 0.
• Figure 4: Graph $G_{180}$ (a) obtained by attaching the four motifs in (b)--(e) via their bottom nodes (color red) in a rotating manner to the cycle of length 12 in the middle. The 28 target classes, denoted by the colors, are defined by the orbits. Top frequent patterns extracted by EEGL from the explanations in rounds 2 (f and g) and 3 (h) for $G_{180}$ in Fig. \ref{['fig:G180']} with their roots $r_1,r_2,r_3$ marked by red. The class predictions obtained by EEGL with vanilla initialization is given in (i)--(l). Each of the 28 colors denotes the same class in (i)--(l).
• Figure 5: Fullerenes $\text{C}_{60}$ (a) and its linegraph (b), $\text{C}_{70}$ (c), and its line graph (d). The colors in (b) and (d) indicate the corresponding bond types in (a) and (c), respectively. The different [6,6], [5,6], and [5,5] types of the bonds (in red) are given in (e)--(m).