Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Coden: Efficient Temporal Graph Neural Networks for Continuous Prediction

Zulun Zhu, Siqiang Luo

Abstract

Temporal Graph Neural Networks (TGNNs) are pivotal in processing dynamic graphs. However, existing TGNNs primarily target one-time predictions for a given temporal span, whereas many practical applications require continuous predictions, that predictions are issued frequently over time. Directly adapting existing TGNNs to continuous-prediction scenarios introduces either significant computational overhead or prediction quality issues especially for large graphs. This paper revisits the challenge of { continuous predictions} in TGNNs, and introduces {\sc Coden}, a TGNN model designed for efficient and effective learning on dynamic graphs. {\sc Coden} innovatively overcomes the key complexity bottleneck in existing TGNNs while preserving comparable predictive accuracy. Moreover, we further provide theoretical analyses that substantiate the effectiveness and efficiency of {\sc Coden}, and clarify its duality relationship with both RNN-based and attention-based models. Our evaluations across five dynamic datasets show that {\sc Coden} surpasses existing performance benchmarks in both efficiency and effectiveness, establishing it as a superior solution for continuous prediction in evolving graph environments.

Coden: Efficient Temporal Graph Neural Networks for Continuous Prediction

Abstract

Temporal Graph Neural Networks (TGNNs) are pivotal in processing dynamic graphs. However, existing TGNNs primarily target one-time predictions for a given temporal span, whereas many practical applications require continuous predictions, that predictions are issued frequently over time. Directly adapting existing TGNNs to continuous-prediction scenarios introduces either significant computational overhead or prediction quality issues especially for large graphs. This paper revisits the challenge of { continuous predictions} in TGNNs, and introduces {\sc Coden}, a TGNN model designed for efficient and effective learning on dynamic graphs. {\sc Coden} innovatively overcomes the key complexity bottleneck in existing TGNNs while preserving comparable predictive accuracy. Moreover, we further provide theoretical analyses that substantiate the effectiveness and efficiency of {\sc Coden}, and clarify its duality relationship with both RNN-based and attention-based models. Our evaluations across five dynamic datasets show that {\sc Coden} surpasses existing performance benchmarks in both efficiency and effectiveness, establishing it as a superior solution for continuous prediction in evolving graph environments.
Paper Structure (47 sections, 8 theorems, 46 equations, 7 figures, 7 tables, 3 algorithms)

This paper contains 47 sections, 8 theorems, 46 equations, 7 figures, 7 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminary and Related Work
  3. Notations
  4. General Message Passing with CTDG
  5. Single-snapshot Methods
  6. RNN-based Methods
  7. Attention-based Methods
  8. State Space Models
  9. Methodology
  10. Overview
  11. Efficient State Update
  12. Efficient Batch Processing
  13. Embedding Update Guaranteeing ||H(t)-Z(t)||_1 <= n(t) epsilon
  14. Complexity Analysis
  15. RNN–Attention Duality in Coden
  16. ...and 32 more sections

Key Result

Lemma 1

If there exists an edge update at time $t+1$ and $||{\bm H}^{(t)} - {\bm Z}^{(t)}||_1 \leq n^{(t)}\epsilon$ holds for time $t$, the difference between ${\bm H}^{(t)}$ and ${\bm H}^{(t+1)}$ satisfies: where ${\bm x}_{max}$ is the row-wise maximum absolute value vector and the $i$-th entry of ${\bm x}_{max}$ is defined as: $\{{\bm x}_{max}\}_i = \max_{1\leq j\leq F}|{\bm X}^{(t)}_{ij}|$.

Figures (7)

  • Figure 1: The comparison between different TGNN paradigms, where each color denotes a different status of the graph.
  • Figure 2: An illustration of the Coden framework. ${\bm M}^{(t)}$, ${\bm H}^{(t)}$ and ${\bm Y}^{(t)}$ denote the node state matrix, embedding matrix and the prediction results at time $t$, respectively.
  • Figure 3: The equivalent kernel attention in Coden.
  • Figure 4: Micro-F1 scores for each prediction time of four datasets.
  • Figure 5: Average training time (per epoch) for each prediction time.
  • ...and 2 more figures

Theorems & Definitions (16)

  • Lemma 1
  • Proposition 1
  • Lemma 2
  • Lemma 3
  • Lemma 4: Gated-RNN View under Constrained Parameters
  • Lemma 5
  • proof
  • proof
  • proof
  • proof
  • ...and 6 more