Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interpretable Prototype-based Graph Information Bottleneck

Sangwoo Seo, Sungwon Kim, Chanyoung Park

TL;DR

This work proposes a novel framework of explainable GNNs, called interpretable Prototype-based Graph Information Bottleneck (PGIB) that incorporates prototype learning within the information bottleneck framework to provide prototypes with the key subgraph from the input graph that is important for the model prediction.

Abstract

The success of Graph Neural Networks (GNNs) has led to a need for understanding their decision-making process and providing explanations for their predictions, which has given rise to explainable AI (XAI) that offers transparent explanations for black-box models. Recently, the use of prototypes has successfully improved the explainability of models by learning prototypes to imply training graphs that affect the prediction. However, these approaches tend to provide prototypes with excessive information from the entire graph, leading to the exclusion of key substructures or the inclusion of irrelevant substructures, which can limit both the interpretability and the performance of the model in downstream tasks. In this work, we propose a novel framework of explainable GNNs, called interpretable Prototype-based Graph Information Bottleneck (PGIB) that incorporates prototype learning within the information bottleneck framework to provide prototypes with the key subgraph from the input graph that is important for the model prediction. This is the first work that incorporates prototype learning into the process of identifying the key subgraphs that have a critical impact on the prediction performance. Extensive experiments, including qualitative analysis, demonstrate that PGIB outperforms state-of-the-art methods in terms of both prediction performance and explainability.

Interpretable Prototype-based Graph Information Bottleneck

TL;DR

This work proposes a novel framework of explainable GNNs, called interpretable Prototype-based Graph Information Bottleneck (PGIB) that incorporates prototype learning within the information bottleneck framework to provide prototypes with the key subgraph from the input graph that is important for the model prediction.

Abstract

The success of Graph Neural Networks (GNNs) has led to a need for understanding their decision-making process and providing explanations for their predictions, which has given rise to explainable AI (XAI) that offers transparent explanations for black-box models. Recently, the use of prototypes has successfully improved the explainability of models by learning prototypes to imply training graphs that affect the prediction. However, these approaches tend to provide prototypes with excessive information from the entire graph, leading to the exclusion of key substructures or the inclusion of irrelevant substructures, which can limit both the interpretability and the performance of the model in downstream tasks. In this work, we propose a novel framework of explainable GNNs, called interpretable Prototype-based Graph Information Bottleneck (PGIB) that incorporates prototype learning within the information bottleneck framework to provide prototypes with the key subgraph from the input graph that is important for the model prediction. This is the first work that incorporates prototype learning into the process of identifying the key subgraphs that have a critical impact on the prediction performance. Extensive experiments, including qualitative analysis, demonstrate that PGIB outperforms state-of-the-art methods in terms of both prediction performance and explainability.
Paper Structure (29 sections, 1 theorem, 30 equations, 12 figures, 4 tables, 1 algorithm)

This paper contains 29 sections, 1 theorem, 30 equations, 12 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Methodology
  4. Prototype-based Graph Information Bottleneck
  5. Subgraph Extraction Layer (Minimizing $I(\mathcal{G} ; \mathcal{G}_{sub})$)
  6. Prototype Layer (Minimizing $-I(Y; \mathcal{G}_{sub}, \mathcal{G}_{p}) + I(Y;\mathcal{G}_{p}|\mathcal{G}_{sub})$ )
  7. Minimizing $-I(Y; \mathcal{G}_{sub}, \mathcal{G}_{p})$
  8. Minimizing $I(Y;\mathcal{G}_{p}|\mathcal{G}_{sub})$
  9. Prediction Layer
  10. Interpretability Stabilization
  11. Experiments
  12. Experimental Settings
  13. Graph Classification
  14. Graph Interpretation
  15. Hyperparameter Analysis
  16. ...and 14 more sections

Key Result

Proposition 1

(Lower bound of $I(Y; \mathcal{G}_{sub}, \mathcal{G}_{p})$) Given significant subgraph $\mathcal{G}_{sub}$ for a graph $\mathcal{G}$, its label information $Y$, prototype graph $\mathcal{G}_{p}$ and similarity function $\gamma$, we have where $q_\theta \left( Y | \gamma \left( \mathcal{G}_{sub}, \mathcal{G}_{p} \right) \right)$ is the variational approximation to the true posterior $p \left( Y |

Figures (12)

  • Figure 1: Comparison of the learned prototypes between ProtGNN and PGIB.
  • Figure 2: The architecture of our proposed PGIB. PGIB generates a subgraph $\mathcal{G}_{sub}$ by injecting noise to identify core subgraphs, and it is used to compute similarity scores between prototypes in the prototype layer. The trained prototypes play a crucial role in visualizing the reasoning processes during training in an interpretable manner. PGIB also involves merging pairs of similar prototypes to decrease the number of prototypes. Finally, the integrated prototypes are utilized to predict the graph labels in the fully connected layer.
  • Figure 3: Explanation visualizations on Mutag (a-d) and BA-2Motifs (e-h)
  • Figure 4: Comparisons of fidelity scores over sparsity scores $k$.
  • Figure 5: $\mathcal{G}_p$ visualization over $\alpha_2$.
  • ...and 7 more figures

Theorems & Definitions (1)

  • Proposition 1