Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IMPaCT GNN: Imposing invariance with Message Passing in Chronological split Temporal Graphs

Sejun Park, Joo Young Park, Hyunwoo Park

TL;DR

This paper proposes Imposing invariance with Message Passing in Chronological split Temporal Graphs (IMPaCT), a method that imposes invariant properties based on realistic assumptions derived from temporal graph structures, and introduces the Temporal Stochastic Block Model (TSBM), which replicates temporal graphs under varying conditions.

Abstract

This paper addresses domain adaptation challenges in graph data resulting from chronological splits. In a transductive graph learning setting, where each node is associated with a timestamp, we focus on the task of Semi-Supervised Node Classification (SSNC), aiming to classify recent nodes using labels of past nodes. Temporal dependencies in node connections create domain shifts, causing significant performance degradation when applying models trained on historical data into recent data. Given the practical relevance of this scenario, addressing domain adaptation in chronological split data is crucial, yet underexplored. We propose Imposing invariance with Message Passing in Chronological split Temporal Graphs (IMPaCT), a method that imposes invariant properties based on realistic assumptions derived from temporal graph structures. Unlike traditional domain adaptation approaches which rely on unverifiable assumptions, IMPaCT explicitly accounts for the characteristics of chronological splits. The IMPaCT is further supported by rigorous mathematical analysis, including a derivation of an upper bound of the generalization error. Experimentally, IMPaCT achieves a 3.8% performance improvement over current SOTA method on the ogbn-mag graph dataset. Additionally, we introduce the Temporal Stochastic Block Model (TSBM), which replicates temporal graphs under varying conditions, demonstrating the applicability of our methods to general spatial GNNs.

IMPaCT GNN: Imposing invariance with Message Passing in Chronological split Temporal Graphs

TL;DR

This paper proposes Imposing invariance with Message Passing in Chronological split Temporal Graphs (IMPaCT), a method that imposes invariant properties based on realistic assumptions derived from temporal graph structures, and introduces the Temporal Stochastic Block Model (TSBM), which replicates temporal graphs under varying conditions.

Abstract

This paper addresses domain adaptation challenges in graph data resulting from chronological splits. In a transductive graph learning setting, where each node is associated with a timestamp, we focus on the task of Semi-Supervised Node Classification (SSNC), aiming to classify recent nodes using labels of past nodes. Temporal dependencies in node connections create domain shifts, causing significant performance degradation when applying models trained on historical data into recent data. Given the practical relevance of this scenario, addressing domain adaptation in chronological split data is crucial, yet underexplored. We propose Imposing invariance with Message Passing in Chronological split Temporal Graphs (IMPaCT), a method that imposes invariant properties based on realistic assumptions derived from temporal graph structures. Unlike traditional domain adaptation approaches which rely on unverifiable assumptions, IMPaCT explicitly accounts for the characteristics of chronological splits. The IMPaCT is further supported by rigorous mathematical analysis, including a derivation of an upper bound of the generalization error. Experimentally, IMPaCT achieves a 3.8% performance improvement over current SOTA method on the ogbn-mag graph dataset. Additionally, we introduce the Temporal Stochastic Block Model (TSBM), which replicates temporal graphs under varying conditions, demonstrating the applicability of our methods to general spatial GNNs.

Paper Structure

This paper contains 56 sections, 12 theorems, 85 equations, 9 figures, 7 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Graph Neural Networks
  4. Domain Adaptation
  5. Prior Studies
  6. Method Explanation
  7. Motivation of Our Method: Imposing Invariance with Message Passing
  8. Problem Setting
  9. Outline of $\textrm{IMPaCT}$ Methods
  10. First moment alignment methods
  11. Persistent Message Passing: $\textrm{PMP}$
  12. Mono-directional Message Passing: $\textrm{MMP}$
  13. Theoretical Analysis of $\textrm{PMP}$ when Applied in Multi-Layer GNNs.
  14. Generalized $\textrm{PMP}$: $\textrm{GenPMP}$
  15. Second moment alignment methods
  16. ...and 41 more sections

Key Result

Theorem 4.1

The 1st moment of aggregated message obtained by $\textrm{PMP}$ layer is invariant, if the 1st moment of previous representation is invariant. Sketch of proof Let $\mathbb{E}_{X\sim{x_{\tilde{y} \tilde{t}}^{(k)}}}\left[X\right]=\mu_{X}^{(k)}(\tilde{y})$ as a function invariant with $\tilde{t}$. Then which is invariant with $t$. See Appendix apdx:PMP for details and implementation.

Figures (9)

  • Figure 1: Illustrative explanation of chronological split dataset.
  • Figure 2: Graphical representation of functions $f$ and $g$. The shaded bars denote relative connectivity. Target node has label $y$, and only consider cases neighboring nodes with a labels $\tilde{y}$. The function $g(y,\tilde{y},|\tilde{t}-t|)$ determines extent to which relative connectivity varies, and its scale is adjusted by the function $f(y, t)$.
  • Figure 3: Graphical explanation of $\textrm{PMP}$
  • Figure 4: The left graphs show the performance gain of $\textrm{IMPaCT}$ over the baseline. The right graphs illustrate the gain of 2nd moment alignment methods over the 1st moment alignment method $\textrm{PMP}$.
  • Figure 5: 2D projection of each community's mean feature by t-SNE. Points corresponding to communities with the same label are represented in the same color. [Left] Plot for all 349 labels, [Right] Plot for 15 labels with the most nodes.
  • ...and 4 more figures

Theorems & Definitions (21)

  • Definition 4.1
  • Theorem 4.1
  • Definition 4.2
  • Theorem 4.2
  • Theorem 4.3
  • Theorem 4.4
  • Theorem 4.5
  • Corollary 4.5.1
  • Definition 4.3
  • Definition 5.1
  • ...and 11 more