Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Structure and Feature Extrapolation for Out-of-Distribution Generalization

Xiner Li, Shurui Gui, Youzhi Luo, Shuiwang Ji

TL;DR

This work tackles graph out-of-distribution generalization by introducing environment-aware non-Euclidean linear extrapolation to synthesize OOD data. It presents G-Splice for structural extrapolation via graph splicing with a bridge generator and FeatX for feature extrapolation via learned non-causal feature perturbations, underpinned by causal-additivity theory. Empirical results across multiple structure-shift and feature-shift benchmarks show that G-Splice and FeatX substantially improve OOD performance, with their combination achieving strong results on complex shifts such as FSMotif. The approach emphasizes the value of explicitly generating OOD graphs and features with environment information, and points to directions for future work including non-linear extrapolation and improved environment discovery.

Abstract

Out-of-distribution (OOD) generalization deals with the prevalent learning scenario where test distribution shifts from training distribution. With rising application demands and inherent complexity, graph OOD problems call for specialized solutions. While data-centric methods exhibit performance enhancements on many generic machine learning tasks, there is a notable absence of data augmentation methods tailored for graph OOD generalization. In this work, we propose to achieve graph OOD generalization with the novel design of non-Euclidean-space linear extrapolation. The proposed augmentation strategy extrapolates both structure and feature spaces to generate OOD graph data. Our design tailors OOD samples for specific shifts without corrupting underlying causal mechanisms. Theoretical analysis and empirical results evidence the effectiveness of our method in solving target shifts, showing substantial and constant improvements across various graph OOD tasks.

Graph Structure and Feature Extrapolation for Out-of-Distribution Generalization

TL;DR

This work tackles graph out-of-distribution generalization by introducing environment-aware non-Euclidean linear extrapolation to synthesize OOD data. It presents G-Splice for structural extrapolation via graph splicing with a bridge generator and FeatX for feature extrapolation via learned non-causal feature perturbations, underpinned by causal-additivity theory. Empirical results across multiple structure-shift and feature-shift benchmarks show that G-Splice and FeatX substantially improve OOD performance, with their combination achieving strong results on complex shifts such as FSMotif. The approach emphasizes the value of explicitly generating OOD graphs and features with environment information, and points to directions for future work including non-linear extrapolation and improved environment discovery.

Abstract

Out-of-distribution (OOD) generalization deals with the prevalent learning scenario where test distribution shifts from training distribution. With rising application demands and inherent complexity, graph OOD problems call for specialized solutions. While data-centric methods exhibit performance enhancements on many generic machine learning tasks, there is a notable absence of data augmentation methods tailored for graph OOD generalization. In this work, we propose to achieve graph OOD generalization with the novel design of non-Euclidean-space linear extrapolation. The proposed augmentation strategy extrapolates both structure and feature spaces to generate OOD graph data. Our design tailors OOD samples for specific shifts without corrupting underlying causal mechanisms. Theoretical analysis and empirical results evidence the effectiveness of our method in solving target shifts, showing substantial and constant improvements across various graph OOD tasks.
Paper Structure (39 sections, 4 theorems, 49 equations, 6 figures, 8 tables)

This paper contains 39 sections, 4 theorems, 49 equations, 6 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Problem Setting
  3. Linear Extrapolation in Graph Space
  4. Causal Analysis
  5. Linear Extrapolation Formulation
  6. Linear Extrapolation for OOD Generalization
  7. G-Splice for Structural Linear Extrapolation
  8. Graph Splice
  9. Component Graph Selection
  10. Post-sampling Procedure
  11. FeatX for Feature Linear Extrapolation
  12. Experimental Studies
  13. OOD Performance on Structure Shifts
  14. OOD Evaluation for Structure and Feature Extrapolation
  15. Discussion and Conclusion
  16. ...and 24 more sections

Key Result

Theorem 3.6

Given an $N$-sample training dataset $D\{{G}_{tr}\}$, its N-dimension structural linear extrapolation can generate sets $D\{{G}_1\}$ and $D\{{G}_2\}$ such that where $<$ denotes "smaller in size" and "lower base complexity" for size and base extrapolation.

Figures (6)

  • Figure 1: Illustration of the causal graph. Unobservable variables lie in the latent space; partially observable variables can be learned and selected.
  • Figure 2: Architecture overview. Causal features are selected and then preserved along with graph topological structure, while extrapolation on non-causal features spans their feature space to solve feature shifts. For structure shifts, diverse component graphs are extracted with pair-learning approaches, selected and spliced with extrapolation strategies to achieve causally-valid structural OOD samples.
  • Figure 3: ROC-AUC score curve and loss curve for GOODHIV-size.
  • Figure 4: Accuracy score curve and loss curve for GOODMotif-size.
  • Figure 5: Accuracy score curve and loss curve for GOODCMNIST-color.
  • ...and 1 more figures

Theorems & Definitions (12)

  • Definition 3.1
  • Definition 3.2
  • Definition 3.4
  • Definition 3.5
  • Theorem 3.6
  • Theorem 3.7
  • Theorem 4.1
  • Theorem 4.2
  • proof
  • proof
  • ...and 2 more