Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Graph Out-Of-Distribution Generalization: A Learnable Random Walk Perspective

Henan Sun, Xunkai Li, Lei Zhu, Junyi Han, Guang Zeng, Ronghua Li, Guoren Wang

TL;DR

The paper tackles graph OOD generalization by challenging the idea that invariant topology or spectrum reliably persists under real-world shifts. It introduces LRW-OOD, which learns invariant random-walk sequences via an OOD-aware LRW encoder and a KDE-based mutual-information loss that enforces sufficiency and invariance across environments. The authors provide theoretical connections between the LRW optimization and the graph OOD objective, along with computational guarantees, and demonstrate strong empirical performance across seven datasets with various distribution shifts, achieving a notable average gain over nine baselines. The approach leverages learnable transition matrices derived from node embeddings and integrates topology and features, offering practical robustness for node-level GNN tasks in non-i.i.d. settings.

Abstract

Out-Of-Distribution (OOD) generalization has gained increasing attentions for machine learning on graphs, as graph neural networks (GNNs) often exhibit performance degradation under distribution shifts. Existing graph OOD methods tend to follow the basic ideas of invariant risk minimization and structural causal models, interpreting the invariant knowledge across datasets under various distribution shifts as graph topology or graph spectrum. However, these interpretations may be inconsistent with real-world scenarios, as neither invariant topology nor spectrum is assured. In this paper, we advocate the learnable random walk (LRW) perspective as the instantiation of invariant knowledge, and propose LRW-OOD to realize graph OOD generalization learning. Instead of employing fixed probability transition matrix (i.e., degree-normalized adjacency matrix), we parameterize the transition matrix with an LRW-sampler and a path encoder. Furthermore, we propose the kernel density estimation (KDE)-based mutual information (MI) loss to generate random walk sequences that adhere to OOD principles. Extensive experiment demonstrates that our model can effectively enhance graph OOD generalization under various types of distribution shifts and yield a significant accuracy improvement of 3.87% over state-of-the-art graph OOD generalization baselines.

Rethinking Graph Out-Of-Distribution Generalization: A Learnable Random Walk Perspective

TL;DR

The paper tackles graph OOD generalization by challenging the idea that invariant topology or spectrum reliably persists under real-world shifts. It introduces LRW-OOD, which learns invariant random-walk sequences via an OOD-aware LRW encoder and a KDE-based mutual-information loss that enforces sufficiency and invariance across environments. The authors provide theoretical connections between the LRW optimization and the graph OOD objective, along with computational guarantees, and demonstrate strong empirical performance across seven datasets with various distribution shifts, achieving a notable average gain over nine baselines. The approach leverages learnable transition matrices derived from node embeddings and integrates topology and features, offering practical robustness for node-level GNN tasks in non-i.i.d. settings.

Abstract

Out-Of-Distribution (OOD) generalization has gained increasing attentions for machine learning on graphs, as graph neural networks (GNNs) often exhibit performance degradation under distribution shifts. Existing graph OOD methods tend to follow the basic ideas of invariant risk minimization and structural causal models, interpreting the invariant knowledge across datasets under various distribution shifts as graph topology or graph spectrum. However, these interpretations may be inconsistent with real-world scenarios, as neither invariant topology nor spectrum is assured. In this paper, we advocate the learnable random walk (LRW) perspective as the instantiation of invariant knowledge, and propose LRW-OOD to realize graph OOD generalization learning. Instead of employing fixed probability transition matrix (i.e., degree-normalized adjacency matrix), we parameterize the transition matrix with an LRW-sampler and a path encoder. Furthermore, we propose the kernel density estimation (KDE)-based mutual information (MI) loss to generate random walk sequences that adhere to OOD principles. Extensive experiment demonstrates that our model can effectively enhance graph OOD generalization under various types of distribution shifts and yield a significant accuracy improvement of 3.87% over state-of-the-art graph OOD generalization baselines.
Paper Structure (24 sections, 4 theorems, 14 equations, 5 figures, 5 tables)

This paper contains 24 sections, 4 theorems, 14 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Formulation
  4. Related works
  5. Methods
  6. Motivation
  7. Model Framework
  8. Theoretical Discussion
  9. Experiments
  10. Experiment Setup
  11. Performance Comparison
  12. Ablation Study
  13. Sensitivity Analysis
  14. Model Visualization
  15. Conclusion
  16. ...and 9 more sections

Key Result

Theorem 3.1

Let $f(\mathbf{G}_e)$ denotes the learnable random walk encoder. If it is optimized by minimizing the KDE-based MI loss defined in Equation eq: kde_mi_loss, then the resulting encoder satisfies both the sufficiency condition: $\textbf{y}=f^*(\textbf{G}_e)+\sigma$ and the invariance condition: $\math

Figures (5)

  • Figure 1: An example of the heterophilic citation network under temporal distribution shift and pipelines of existing graph OOD models. Topology-based and spectrum-based pipelines are the two primary approaches for graph OOD generalization, while the random-walk-based pipeline is the proposed one in this paper.
  • Figure 2: The framework of the proposed LRW-OOD.
  • Figure 3: The performance comparison of graph OOD models using GCN as the backbone.
  • Figure 4: The performance comparison of graph OOD models using GAT as the backbone.
  • Figure 5: The visualization of weights of LRW-OOD on the synthetic datasets.

Theorems & Definitions (5)

  • Theorem 3.1
  • Theorem 3.2
  • Theorem 3.3
  • Lemma A.1
  • proof