Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scalable and Efficient Temporal Graph Representation Learning via Forward Recent Sampling

Yuhong Luo, Pan Li

TL;DR

A novel TGRL framework, No-Looking-Back (NLB), which overcomes challenges by introducing a forward recent sampling strategy and eliminates the need to backtrack through historical interactions by utilizing a GPU-executable, size-constrained hash table for each node.

Abstract

Temporal graph representation learning (TGRL) is essential for modeling dynamic systems in real-world networks. However, traditional TGRL methods, despite their effectiveness, often face significant computational challenges and inference delays due to the inefficient sampling of temporal neighbors. Conventional sampling methods typically involve backtracking through the interaction history of each node. In this paper, we propose a novel TGRL framework, No-Looking-Back (NLB), which overcomes these challenges by introducing a forward recent sampling strategy. This strategy eliminates the need to backtrack through historical interactions by utilizing a GPU-executable, size-constrained hash table for each node. The hash table records a down-sampled set of recent interactions, enabling rapid query responses with minimal inference latency. The maintenance of this hash table is highly efficient, operating with $O(1)$ complexity. Fully compatible with GPU processing, NLB maximizes programmability, parallelism, and power efficiency. Empirical evaluations demonstrate that NLB not only matches or surpasses state-of-the-art methods in accuracy for tasks like link prediction and node classification across six real-world datasets but also achieves 1.32-4.40x faster training, 1.2-7.94x greater energy efficiency, and 1.63-12.95x lower inference latency compared to competitive baselines. The link to the code: https://github.com/Graph-COM/NLB.

Scalable and Efficient Temporal Graph Representation Learning via Forward Recent Sampling

TL;DR

A novel TGRL framework, No-Looking-Back (NLB), which overcomes challenges by introducing a forward recent sampling strategy and eliminates the need to backtrack through historical interactions by utilizing a GPU-executable, size-constrained hash table for each node.

Abstract

Temporal graph representation learning (TGRL) is essential for modeling dynamic systems in real-world networks. However, traditional TGRL methods, despite their effectiveness, often face significant computational challenges and inference delays due to the inefficient sampling of temporal neighbors. Conventional sampling methods typically involve backtracking through the interaction history of each node. In this paper, we propose a novel TGRL framework, No-Looking-Back (NLB), which overcomes these challenges by introducing a forward recent sampling strategy. This strategy eliminates the need to backtrack through historical interactions by utilizing a GPU-executable, size-constrained hash table for each node. The hash table records a down-sampled set of recent interactions, enabling rapid query responses with minimal inference latency. The maintenance of this hash table is highly efficient, operating with complexity. Fully compatible with GPU processing, NLB maximizes programmability, parallelism, and power efficiency. Empirical evaluations demonstrate that NLB not only matches or surpasses state-of-the-art methods in accuracy for tasks like link prediction and node classification across six real-world datasets but also achieves 1.32-4.40x faster training, 1.2-7.94x greater energy efficiency, and 1.63-12.95x lower inference latency compared to competitive baselines. The link to the code: https://github.com/Graph-COM/NLB.
Paper Structure (13 sections, 1 theorem, 4 equations, 2 figures, 4 tables)

This paper contains 13 sections, 1 theorem, 4 equations, 2 figures, 4 tables.

Table of Contents

  1. Notations and Problem Formulation
  2. Methodology
  3. The sampling strategies of existing works
  4. Our approach: Forward recent sampling
  5. The properties of forward recent sampling
  6. Node representation generation and NLB prediction
  7. Experiments
  8. Experimental setup
  9. Results and Discussion
  10. Additional analysis
  11. Limitations and Future Work
  12. Conclusion
  13. Acknowledgement

Key Result

Theorem 4.4

Suppose links come in for any node (e.g. $u$) by following a Poisson point process with a constant intensity (e.g. $\lambda$), and suppose a temporal neighbor $(v, e_{u,v}^{t_i},t_i)$ is inserted into $S_u$ at time $t_i$, then $\Pr((v,e_{u,v}^{t_i},t_i) \in S_u^t) = \exp(\frac{\alpha\lambda}{s}(t_i

Figures (2)

  • Figure 2: The transductive link prediction validation performance v.s. training time on Reddit. Each dot on the curves gets collected at the end of an epoch.
  • Figure 3: The changes in transductive link prediction test performance on Reddit (Left) and in inductive link prediction test performance on Ubuntu (Right) with respect to $\alpha$'s.

Theorems & Definitions (7)

  • Definition 3.1: Temporal network
  • Definition 3.2: Temporal neighbors of a node
  • Definition 3.3: Problem formulation
  • Definition 4.1: Truncation
  • Definition 4.2: Uniform sampling
  • Definition 4.3: Recent sampling
  • Theorem 4.4: NLB-edge achieves recent sampling