Dissecting the Failure of Invariant Learning on Graphs
Qixun Wang, Yifei Wang, Yisen Wang, Xianghua Ying
TL;DR
This work reveals that traditional invariant learning methods like IRM and VREx often fail for node-level OOD generalization on graphs due to neglecting class-conditional invariance in the graph setting. The authors introduce a Structural Causal Model to dissect how invariant ego-graphs and spurious features interact with graph structure, and propose Cross-environment Intra-class Alignment (CIA) to enforce class-aware invariance, plus CIA-LRA for environments without labels by leveraging local neighborhood label distributions. They provide a PAC-Bayesian OOD generalization bound under a CSBM-OOD framework and empirically demonstrate that CIA achieves superior graph-OOD generalization, with CIA-LRA delivering further improvements and enabling plug-in gains for existing graph-OOD methods. The results offer a principled path to robust node-level OOD learning on graphs and highlight the importance of locality and neighborhood distribution shifts in graph data.
Abstract
Enhancing node-level Out-Of-Distribution (OOD) generalization on graphs remains a crucial area of research. In this paper, we develop a Structural Causal Model (SCM) to theoretically dissect the performance of two prominent invariant learning methods -- Invariant Risk Minimization (IRM) and Variance-Risk Extrapolation (VREx) -- in node-level OOD settings. Our analysis reveals a critical limitation: due to the lack of class-conditional invariance constraints, these methods may struggle to accurately identify the structure of the predictive invariant ego-graph and consequently rely on spurious features. To address this, we propose Cross-environment Intra-class Alignment (CIA), which explicitly eliminates spurious features by aligning cross-environment representations conditioned on the same class, bypassing the need for explicit knowledge of the causal pattern structure. To adapt CIA to node-level OOD scenarios where environment labels are hard to obtain, we further propose CIA-LRA (Localized Reweighting Alignment) that leverages the distribution of neighboring labels to selectively align node representations, effectively distinguishing and preserving invariant features while removing spurious ones, all without relying on environment labels. We theoretically prove CIA-LRA's effectiveness by deriving an OOD generalization error bound based on PAC-Bayesian analysis. Experiments on graph OOD benchmarks validate the superiority of CIA and CIA-LRA, marking a significant advancement in node-level OOD generalization. The codes are available at https://github.com/NOVAglow646/NeurIPS24-Invariant-Learning-on-Graphs.