Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing the Expressivity of Temporal Graph Networks through Source-Target Identification

Benedict Aaron Tjandra, Federico Barbero, Michael Bronstein

TL;DR

The paper addresses the gap where Temporal Graph Networks underperform on dynamic node affinity prediction compared to simple heuristics. It proves that standard TGNs cannot represent moving averages or autoregressive models over messages and introduces TGNv2, which adds source-target identifiers to messages to break permutation-invariance and enable richer temporal reasoning. Theoretical results show TGNv2 is strictly more expressive, and empirical results on the Temporal Graph Benchmark demonstrate significant performance gains over existing TG models, though heuristics remain strong baselines. Overall, the work provides a principled approach to increasing the expressivity of TGNs for dynamic interaction forecasting with potential applications in recommender systems and international trade modeling.

Abstract

Despite the successful application of Temporal Graph Networks (TGNs) for tasks such as dynamic node classification and link prediction, they still perform poorly on the task of dynamic node affinity prediction -- where the goal is to predict 'how much' two nodes will interact in the future. In fact, simple heuristic approaches such as persistent forecasts and moving averages over ground-truth labels significantly and consistently outperform TGNs. Building on this observation, we find that computing heuristics over messages is an equally competitive approach, outperforming TGN and all current temporal graph (TG) models on dynamic node affinity prediction. In this paper, we prove that no formulation of TGN can represent persistent forecasting or moving averages over messages, and propose to enhance the expressivity of TGNs by adding source-target identification to each interaction event message. We show that this modification is required to represent persistent forecasting, moving averages, and the broader class of autoregressive models over messages. Our proposed method, TGNv2, significantly outperforms TGN and all current TG models on all Temporal Graph Benchmark (TGB) dynamic node affinity prediction datasets.

Enhancing the Expressivity of Temporal Graph Networks through Source-Target Identification

TL;DR

The paper addresses the gap where Temporal Graph Networks underperform on dynamic node affinity prediction compared to simple heuristics. It proves that standard TGNs cannot represent moving averages or autoregressive models over messages and introduces TGNv2, which adds source-target identifiers to messages to break permutation-invariance and enable richer temporal reasoning. Theoretical results show TGNv2 is strictly more expressive, and empirical results on the Temporal Graph Benchmark demonstrate significant performance gains over existing TG models, though heuristics remain strong baselines. Overall, the work provides a principled approach to increasing the expressivity of TGNs for dynamic interaction forecasting with potential applications in recommender systems and international trade modeling.

Abstract

Despite the successful application of Temporal Graph Networks (TGNs) for tasks such as dynamic node classification and link prediction, they still perform poorly on the task of dynamic node affinity prediction -- where the goal is to predict 'how much' two nodes will interact in the future. In fact, simple heuristic approaches such as persistent forecasts and moving averages over ground-truth labels significantly and consistently outperform TGNs. Building on this observation, we find that computing heuristics over messages is an equally competitive approach, outperforming TGN and all current temporal graph (TG) models on dynamic node affinity prediction. In this paper, we prove that no formulation of TGN can represent persistent forecasting or moving averages over messages, and propose to enhance the expressivity of TGNs by adding source-target identification to each interaction event message. We show that this modification is required to represent persistent forecasting, moving averages, and the broader class of autoregressive models over messages. Our proposed method, TGNv2, significantly outperforms TGN and all current TG models on all Temporal Graph Benchmark (TGB) dynamic node affinity prediction datasets.

Paper Structure

This paper contains 24 sections, 6 theorems, 32 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. The Hidden Limitation of Temporal Graph Networks
  4. TGNv2: Increasing the expressive power of TGNs
  5. Experiments
  6. Related Work
  7. Conclusion
  8. TGN Recap
  9. Message Function.
  10. Message Aggregator.
  11. Memory Updater.
  12. Embedding
  13. Proofs
  14. Proof for Theorem 1
  15. Proof for Corollary 2
  16. ...and 9 more sections

Key Result

Theorem 1

No formulation of TGN can represent a moving average of order $k \in \mathbb{N}^+$ for any temporal graph with a bounded number of vertices.

Figures (1)

  • Figure 1: Graphs $G$ and $G'$. Clearly Node 1 in $G$ and $G'$ are sending different sequences of messages, but TGNs are unable to distinguish them.

Theorems & Definitions (9)

  • Theorem 1
  • Corollary 1
  • Theorem 2
  • Theorem 1
  • proof
  • Corollary 2
  • proof
  • Theorem 2
  • proof