Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalizing Graph Neural Networks on Out-Of-Distribution Graphs

Shaohua Fan, Xiao Wang, Chuan Shi, Peng Cui, Bai Wang

TL;DR

This paper argues that the spurious correlation exists among subgraph-level units and analyzes the degeneration of GNN in causal view, and proposes a general causal representation framework for stable GNN, called StableGNN, which not only outperforms the state-of-the-arts but also provides a flexible framework to enhance existing GNNs.

Abstract

Graph Neural Networks (GNNs) are proposed without considering the agnostic distribution shifts between training and testing graphs, inducing the degeneration of the generalization ability of GNNs on Out-Of-Distribution (OOD) settings. The fundamental reason for such degeneration is that most GNNs are developed based on the I.I.D hypothesis. In such a setting, GNNs tend to exploit subtle statistical correlations existing in the training set for predictions, even though it is a spurious correlation. However, such spurious correlations may change in testing environments, leading to the failure of GNNs. Therefore, eliminating the impact of spurious correlations is crucial for stable GNNs. To this end, we propose a general causal representation framework, called StableGNN. The main idea is to extract high-level representations from graph data first and resort to the distinguishing ability of causal inference to help the model get rid of spurious correlations. Particularly, we exploit a graph pooling layer to extract subgraph-based representations as high-level representations. Furthermore, we propose a causal variable distinguishing regularizer to correct the biased training distribution. Hence, GNNs would concentrate more on the stable correlations. Extensive experiments on both synthetic and real-world OOD graph datasets well verify the effectiveness, flexibility and interpretability of the proposed framework.

Generalizing Graph Neural Networks on Out-Of-Distribution Graphs

TL;DR

This paper argues that the spurious correlation exists among subgraph-level units and analyzes the degeneration of GNN in causal view, and proposes a general causal representation framework for stable GNN, called StableGNN, which not only outperforms the state-of-the-arts but also provides a flexible framework to enhance existing GNNs.

Abstract

Graph Neural Networks (GNNs) are proposed without considering the agnostic distribution shifts between training and testing graphs, inducing the degeneration of the generalization ability of GNNs on Out-Of-Distribution (OOD) settings. The fundamental reason for such degeneration is that most GNNs are developed based on the I.I.D hypothesis. In such a setting, GNNs tend to exploit subtle statistical correlations existing in the training set for predictions, even though it is a spurious correlation. However, such spurious correlations may change in testing environments, leading to the failure of GNNs. Therefore, eliminating the impact of spurious correlations is crucial for stable GNNs. To this end, we propose a general causal representation framework, called StableGNN. The main idea is to extract high-level representations from graph data first and resort to the distinguishing ability of causal inference to help the model get rid of spurious correlations. Particularly, we exploit a graph pooling layer to extract subgraph-based representations as high-level representations. Furthermore, we propose a causal variable distinguishing regularizer to correct the biased training distribution. Hence, GNNs would concentrate more on the stable correlations. Extensive experiments on both synthetic and real-world OOD graph datasets well verify the effectiveness, flexibility and interpretability of the proposed framework.
Paper Structure (15 sections, 2 theorems, 19 equations, 9 figures, 5 tables, 1 algorithm)

This paper contains 15 sections, 2 theorems, 19 equations, 9 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Graph Neural Networks
  4. Causal Representation Learning
  5. Stable Learning
  6. Problem Formulation
  7. The Proposed Method
  8. Overview
  9. Graph High-level Variable Learning
  10. Causal Variable Distinguishing Regularizer
  11. Weighted Graph Neural Networks
  12. Experiments
  13. Experiments on Synthetic Data
  14. Experiments on Real-world Datasets
  15. Conclusion and Future work

Key Result

Lemma 1

Permutation Invariance ying2018hierarchical. Given any permutation matrix $\mathbf{P}\in\{0,1\}^{n\times n}$, if $\mathbf{P}\cdot\text{GNN}(\mathbf{A},\mathbf{F})=\text{GNN}(\mathbf{P}\mathbf{A}\mathbf{P}^\mathrm{T}, \mathbf{P}\mathbf{F})$ (i.e., the GNN method used is permutation equivariant), then

Figures (9)

  • Figure 1: Visualization of subgraph importance for "house" motif classification task, produced by the vanilla GCN model and StableGNN when most of training graphs containing "house" motifs with "star" motifs. The red subgraph indicates the most important subgraph used by the model for prediction (generated by GNNExplainer ying2019gnnexplainer). Due to the spurious correlation, the GCN model tends to focus more on "star" motifs while our model focuses mostly on "house" motifs. For more cases of testing graphs, please refer to Figure \ref{['fig:syn_explainer']}.
  • Figure 2: Causal graph for data generation process. Gray nodes and white nodes mean the unobserved latent variables and the observed variables, respectively.
  • Figure 3: The overall architecture of the proposed StableGNN.
  • Figure 4: Causal view on GNNs.
  • Figure 5: Explanation cases of GCN and StableGCN. Red nodes are the important subgraph calculated by the GNNExplainer.
  • ...and 4 more figures

Theorems & Definitions (4)

  • Lemma 1
  • proof
  • Theorem 1
  • proof