Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ScaDyG:A New Paradigm for Large-scale Dynamic Graph Learning

Xiang Wu, Xunkai Li, Rong-Hua Li, Kangfei Zhao, Guoren Wang

TL;DR

Extensive experiments on 12datasets demonstrate that ScaDyG performs comparably or even outperforms other SOTA methods in both node and link-level downstream tasks, with fewer learnable parameters and higher efficiency.

Abstract

Dynamic graphs (DGs), which capture time-evolving relationships between graph entities, have widespread real-world applications. To efficiently encode DGs for downstream tasks, most dynamic graph neural networks follow the traditional message-passing mechanism and extend it with time-based techniques. Despite their effectiveness, the growth of historical interactions introduces significant scalability issues, particularly in industry scenarios. To address this limitation, we propose ScaDyG, with the core idea of designing a time-aware scalable learning paradigm as follows: 1) Time-aware Topology Reformulation: ScaDyG first segments historical interactions into time steps (intra and inter) based on dynamic modeling, enabling weight-free and time-aware graph propagation within pre-processing. 2) Dynamic Temporal Encoding: To further achieve fine-grained graph propagation within time steps, ScaDyG integrates temporal encoding through a combination of exponential functions in a scalable manner. 3) Hypernetwork-driven Message Aggregation: After obtaining the propagated features (i.e., messages), ScaDyG utilizes hypernetwork to analyze historical dependencies, implementing node-wise representation by an adaptive temporal fusion. Extensive experiments on 12 datasets demonstrate that ScaDyG performs comparably well or even outperforms other SOTA methods in both node and link-level downstream tasks, with fewer learnable parameters and higher efficiency.

ScaDyG:A New Paradigm for Large-scale Dynamic Graph Learning

TL;DR

Extensive experiments on 12datasets demonstrate that ScaDyG performs comparably or even outperforms other SOTA methods in both node and link-level downstream tasks, with fewer learnable parameters and higher efficiency.

Abstract

Dynamic graphs (DGs), which capture time-evolving relationships between graph entities, have widespread real-world applications. To efficiently encode DGs for downstream tasks, most dynamic graph neural networks follow the traditional message-passing mechanism and extend it with time-based techniques. Despite their effectiveness, the growth of historical interactions introduces significant scalability issues, particularly in industry scenarios. To address this limitation, we propose ScaDyG, with the core idea of designing a time-aware scalable learning paradigm as follows: 1) Time-aware Topology Reformulation: ScaDyG first segments historical interactions into time steps (intra and inter) based on dynamic modeling, enabling weight-free and time-aware graph propagation within pre-processing. 2) Dynamic Temporal Encoding: To further achieve fine-grained graph propagation within time steps, ScaDyG integrates temporal encoding through a combination of exponential functions in a scalable manner. 3) Hypernetwork-driven Message Aggregation: After obtaining the propagated features (i.e., messages), ScaDyG utilizes hypernetwork to analyze historical dependencies, implementing node-wise representation by an adaptive temporal fusion. Extensive experiments on 12 datasets demonstrate that ScaDyG performs comparably well or even outperforms other SOTA methods in both node and link-level downstream tasks, with fewer learnable parameters and higher efficiency.

Paper Structure

This paper contains 23 sections, 1 theorem, 11 equations, 6 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Preliminaries
  4. Dynamic graph representation learning
  5. The Proposed Method
  6. Time-aware Topology Reformulation
  7. Dynamic time encoding
  8. Hypernetwork-driven Message Aggregation
  9. Experiments
  10. Experimental settings
  11. Overall performance
  12. Scalability performance
  13. Interpretability Analysis
  14. Hyperparameter Sensitivity
  15. Conclusion
  16. ...and 8 more sections

Key Result

Proposition 1

If $\Delta t = t-t'$ can be split by $t_s$ as $\Delta t_1 = t_s-t'$, $\Delta t_2$ = $t - t_s$, and $\mathbf{W}_1$ is a learnable parameter matrix, $\mathbf{x}_v\odot T_e(\Delta t_1)\odot T_e(\Delta t_2)\mathbf{W}_1$ is equivalent to $\mathbf{x}_v \kappa (\Delta t) \mathbf{W}$.

Figures (6)

  • Figure 1: Overview of ScaDyG. $\odot$ denotes the element-wise multiplication, and $\otimes$ denotes the outer product. We display the propagation from edge-to-node features in two steps as a toy example.
  • Figure 2: Efficiency comparison of ScaDyG and baselines.
  • Figure 3: On MOOC dataset.
  • Figure 4: On tgbl-trade dataset.
  • Figure 6: Results of hyperparameter study.
  • ...and 1 more figures

Theorems & Definitions (4)

  • Definition 1
  • Definition 2
  • Definition 3
  • Proposition 1