Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EvoluNet: Advancing Dynamic Non-IID Transfer Learning on Graphs

Haohui Wang, Yuzhen Mao, Yujun Yan, Yaoqing Yang, Jianhui Sun, Kevin Choi, Balaji Veeramani, Alison Hu, Edward Bowen, Tyler Cody, Dawei Zhou

TL;DR

This work addresses dynamic non-IID transfer learning on graphs, where source graphs observed over $T$ timestamps inform predictions on a evolving target graph at $T+1$ with limited labels. It derives a novel generalization bound showing the target error $\epsilon_{tgt}^{(T+1)}(h)$ is governed by the minimum historical empirical error, a dynamic Wasserstein distance $\tilde{W}_p$ that captures domain evolution, and the hypothesis complexity term $\tilde{\Re}(\mathcal{H}_{\mathcal{L}})$. Guided by this theory, EvoluNet combines a transformer-based multi-resolution temporal encoding (M1) with dual-divergence unification (M2) to learn domain-invariant representations across evolving graphs. On DBLP and HCP benchmarks, EvoluNet achieves up to $12.1\%$ relative improvements in AUC over strong baselines, demonstrating the practical value of modeling temporal evolution and cross-domain alignment for dynamic graph transfer learning.

Abstract

Non-IID transfer learning on graphs is crucial in many high-stakes domains. The majority of existing works assume stationary distribution for both source and target domains. However, real-world graphs are intrinsically dynamic, presenting challenges in terms of domain evolution and dynamic discrepancy between source and target domains. To bridge the gap, we shift the problem to the dynamic setting and pose the question: given the label-rich source graphs and the label-scarce target graphs both observed in previous T timestamps, how can we effectively characterize the evolving domain discrepancy and optimize the generalization performance of the target domain at the incoming T+1 timestamp? To answer it, we propose a generalization bound for dynamic non-IID transfer learning on graphs, which implies the generalization performance is dominated by domain evolution and domain discrepancy between source and target graphs. Inspired by the theoretical results, we introduce a novel generic framework named EvoluNet. It leverages a transformer-based temporal encoding module to model temporal information of the evolving domains and then uses a dynamic domain unification module to efficiently learn domain-invariant representations across the source and target domains. Finally, EvoluNet outperforms the state-of-the-art models by up to 12.1%, demonstrating its effectiveness in transferring knowledge from dynamic source graphs to dynamic target graphs.

EvoluNet: Advancing Dynamic Non-IID Transfer Learning on Graphs

TL;DR

This work addresses dynamic non-IID transfer learning on graphs, where source graphs observed over timestamps inform predictions on a evolving target graph at with limited labels. It derives a novel generalization bound showing the target error is governed by the minimum historical empirical error, a dynamic Wasserstein distance that captures domain evolution, and the hypothesis complexity term . Guided by this theory, EvoluNet combines a transformer-based multi-resolution temporal encoding (M1) with dual-divergence unification (M2) to learn domain-invariant representations across evolving graphs. On DBLP and HCP benchmarks, EvoluNet achieves up to relative improvements in AUC over strong baselines, demonstrating the practical value of modeling temporal evolution and cross-domain alignment for dynamic graph transfer learning.

Abstract

Non-IID transfer learning on graphs is crucial in many high-stakes domains. The majority of existing works assume stationary distribution for both source and target domains. However, real-world graphs are intrinsically dynamic, presenting challenges in terms of domain evolution and dynamic discrepancy between source and target domains. To bridge the gap, we shift the problem to the dynamic setting and pose the question: given the label-rich source graphs and the label-scarce target graphs both observed in previous T timestamps, how can we effectively characterize the evolving domain discrepancy and optimize the generalization performance of the target domain at the incoming T+1 timestamp? To answer it, we propose a generalization bound for dynamic non-IID transfer learning on graphs, which implies the generalization performance is dominated by domain evolution and domain discrepancy between source and target graphs. Inspired by the theoretical results, we introduce a novel generic framework named EvoluNet. It leverages a transformer-based temporal encoding module to model temporal information of the evolving domains and then uses a dynamic domain unification module to efficiently learn domain-invariant representations across the source and target domains. Finally, EvoluNet outperforms the state-of-the-art models by up to 12.1%, demonstrating its effectiveness in transferring knowledge from dynamic source graphs to dynamic target graphs.
Paper Structure (15 sections, 7 theorems, 37 equations, 6 figures, 4 tables, 1 algorithm)

This paper contains 15 sections, 7 theorems, 37 equations, 6 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Model
  4. Theoretical Analysis
  5. EvoluNet Framework
  6. Experiments
  7. Experiment Setup
  8. Effectiveness
  9. Ablation Study
  10. Parameter Sensitivity Analysis
  11. Related Work
  12. Conclusion
  13. Algorithm Analysis
  14. Optimization and Pseudo Code
  15. Implementation Details

Key Result

Lemma 1

For arbitrary classifier $h$ and loss function $\mathcal{L}$ satisfying Assumption asmp:R-Lip and asmp:rho-Lip, the expected error of $h$ on two arbitrary domain $\mathcal{D}_\mu$ and $\mathcal{D}_\nu$ satisfies where $W_{p}$ is the $p$-Wasserstein distance metric and $p \geq 1$.

Figures (6)

  • Figure 1: A paradigm shift to the dynamic non-IID transfer learning. $\mathcal{D}$ denotes IID domain; $\mathcal{G}$ denotes non-IID graph domain. Subscript $src$ and $tgt$ denote source and target, and superscript $(i)$ represents the $i^\text{th}$ timestamp.
  • Figure 2: An illustrative example of dynamic non-IID transfer learning on book review graph and movie review graph. As an example, consider a new series launched on a movie website, where the original book of this series may have been published for decades. It is very natural to transfer knowledge from the information-rich source domain (book) to the information-scarce target domain (movie) across time in order to solve the target task (movie review prediction) at $\mathcal{G}_{tgt}^{(T+1)}$.
  • Figure 3: The proposed EvoluNet framework.
  • Figure 4: An illustrative example of why multi-resolution temporal encoding is important.
  • Figure 5: The EEE-plots of temporal graphs in Table \ref{['tab:datasets']}.
  • ...and 1 more figures

Theorems & Definitions (15)

  • Definition 1: Dynamic $p$-Wasserstein Distance on Graphs
  • Lemma 1: Error Difference over Shifted Domains Wang22Understanding
  • Lemma 2: Algorithm Stability, from Lemma A.1 in Kumar Kumar20Understanding
  • Theorem 1
  • proof
  • Definition 1: Dynamic $p$-Wasserstein Distance on Graphs
  • Definition 2: $p$-Wasserstein Distance villani09optimal
  • Definition 3: Weisfeiler-Lehman subtree Shervashidze11Weisfeiler
  • Definition 4: Graph Discrepancy wu2023non
  • Definition 5: Rademacher Complexity Bartlett02Rademacher
  • ...and 5 more