RegExplainer: Generating Explanations for Graph Neural Networks in Regression Tasks

TL;DR

This work proposes a novel explanation method to interpret the graph regression models (XAIG-R) and introduces a novel objective based on the graph information bottleneck theory (GIB) and a new mix-up framework, which can support various GNNs and explainers in a model-agnostic manner.

Abstract

Graph regression is a fundamental task that has gained significant attention in various graph learning tasks. However, the inference process is often not easily interpretable. Current explanation techniques are limited to understanding Graph Neural Network (GNN) behaviors in classification tasks, leaving an explanation gap for graph regression models. In this work, we propose a novel explanation method to interpret the graph regression models (XAIG-R). Our method addresses the distribution shifting problem and continuously ordered decision boundary issues that hinder existing methods away from being applied in regression tasks. We introduce a novel objective based on the graph information bottleneck theory (GIB) and a new mix-up framework, which can support various GNNs and explainers in a model-agnostic manner. Additionally, we present a self-supervised learning strategy to tackle the continuously ordered labels in regression tasks. We evaluate our proposed method on three benchmark datasets and a real-life dataset introduced by us, and extensive experiments demonstrate its effectiveness in interpreting GNN models in regression tasks.

Figures (7)

• Figure 1: Intuitive illustration of the distribution shifting problem. The 3-dimensional map represents a trained GNN model $f$, where $(h_1, h_2)$ represents the embedding distribution of the graph in two dimensions, and $Y$ represents the prediction value of the graph through $f$. The red and blue lines represent the distribution of the original training graph set and the corresponding explanation sub-graph set, respectively. The distribution of $G^*$ shifts away from the original distribution, resulting in shifted prediction values.
• Figure 2: Intuitive illustration about why $I(G^*;Y)\geq I(Y^*;Y)$. $G^*$ contains more mutual information as having more overlapping area with $Y$ than the overlapping area between $Y^*$ and $Y$.
• Figure 3: Visualization of distribution shifting problem on four graph regression datasets. The points represent the regression value, where the blue points mean ground truth label $Y$, red points mean prediction $f(G)$, and the green points mean prediction $f(G^*)$ on the four datasets. The x-axis is the indices of the graph, sorted by the value of the label $Y$.
• Figure 4: Illustration of RegExplainer. $G$ is the to-be-explained graph, $G^+$ and $G^-$ are the randomly sampled positive and negative neighbors. The explanation of the graph is produced by the explainer model. Then graph $G^*$ is mixed with $(G^+)^\Delta=G^+-(G^+)^*$ and $(G^-)^\Delta=G^--(G^-)^*$ respectively to produce $G^{\text{(mix)}+}$ and $G^{\text{(mix)}-}$. Then the graphs are fed into the trained GNN model to retrieve the embedding vectors ${\bm{h}}^+$, ${\bm{h}}^-$, ${\bm{h}}^{\text{(mix)}+}$ and ${\bm{h}}^{\text{(mix)}-}$, where ${\bm{h}}^{\text{(mix)}+} \approx {\bm{h}}^{\text{(mix)}-}$ due to the same label-preserving sub-graph $G^*$. We use InfoNCE loss to minimize the distance between $G^{\text{(mix)}+}$ and the positive sample and maximize the distance between $G^{\text{(mix)}-}$ and the negative sample. The explainer is trained with the GIB objective and self-supervised contrastive loss.
• Figure 5: Ablation study of RegExplainer. We evaluated the AUC performance of the original RegExplainer and its variants that exclude the mix-up approach, InfoNCE loss, or MSE loss, respectively. The black solid line shows the standard deviation.

Theorems & Definitions (2)

• proof
• proof