SCL-GNN: Towards Generalizable Graph Neural Networks via Spurious Correlation Learning

TL;DR

The Spurious Correlation Learning Graph Neural Network (SCL-GNN) is proposed, a novel framework designed to enhance generalization on both Independent and Identically Distributed (IID) and Out-of-Distribution (OOD) graphs, and incorporates a principled spurious correlation learning mechanism.

Abstract

Graph Neural Networks (GNNs) have demonstrated remarkable success across diverse tasks. However, their generalization capability is often hindered by spurious correlations between node features and labels in the graph. Our analysis reveals that GNNs tend to exploit imperceptible statistical correlations in training data, even when such correlations are unreliable for prediction. To address this challenge, we propose the Spurious Correlation Learning Graph Neural Network (SCL-GNN), a novel framework designed to enhance generalization on both Independent and Identically Distributed (IID) and Out-of-Distribution (OOD) graphs. SCL-GNN incorporates a principled spurious correlation learning mechanism, leveraging the Hilbert-Schmidt Independence Criterion (HSIC) to quantify correlations between node representations and class scores. This enables the model to identify and mitigate irrelevant but influential spurious correlations effectively. Additionally, we introduce an efficient bi-level optimization strategy to jointly optimize modules and GNN parameters, preventing overfitting. Extensive experiments on real-world and synthetic datasets demonstrate that SCL-GNN consistently outperforms state-of-the-art baselines under various distribution shifts, highlighting its robustness and generalization capabilities.

Figures (6)

• Figure 1: An illustration of node classification task on an academic network for GNNs. (a) The task aims to classify the label $y$ of target node $v$ based on the compact graph $\mathcal{G}_v = \{\mathbf{A}_v, \mathbf{X}_v$} consisting of its neighbors. (b) Four existing relations and the notion in the network example, only the AI researcher feature of the collaborator has a positive influence for predicting that the researcher also researches AI. (c) Description and comparison of the workflow of SCL-GNN and existing works. We virtually split features into spurious and stable sets. We propose a more effective method to achieve the same goal by spurious correlation learning, instead of importing complex probability computation.
• Figure 2: Illustration of SCL-GNN framework for generalizing GNN via spurious correlation learning. (a) and (b) illustrates the details of spurious correlation(abbreviated as SC) learner $f_a$ and backbone GNN model $f_s$, respectively, (c) represent an overview of the framework, (d) and (e) respectively explain the process of spurious correlation learning and generalization via pseudo algorithms. When the training is finished, the $f_a$ is fixed synchronously, and we only need to evaluate GNN model with the fine-tuned model weight $\mathbf{W}^\prime$.
• Figure 3: Model performance with different $\beta$.
• Figure 4: Ablation study of SCL-GNN with GCN and GAT.
• Figure 5: Comparison of a GCN backbone's learning curves

Theorems & Definitions (1)

• Definition 1