Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Expressivity of Representation Learning on Continuous-Time Dynamic Graphs: An Information-Flow Centric Review

Sofiane Ennadir, Gabriela Zarzar Gandler, Filip Cornell, Lele Cao, Oleg Smirnov, Tianze Wang, Levente Zólyomi, Björn Brinne, Sahar Asadi

TL;DR

The paper addresses expressivity in continuous-time dynamic graphs by introducing an information-flow framework that quantifies how events update node representations through memory, temporal projection, and MP components. It categorizes CTDG methods into non-MP, MP-based, and hybrid families, linking architectural choices to their theoretical information propagation bounds, and ties SSRL methods to the same expressivity lens. Empirical validation across synthetic and real datasets demonstrates that MP-based models, especially CTAN and TGN variants, effectively capture long-range and community-structural dynamics, while SSRL pretext tasks benefit from methods with strong propagation capabilities. The work provides a principled roadmap for selecting and designing CTDG models with desired expressivity and data efficiency for diverse graph types and downstream tasks.

Abstract

Graphs are ubiquitous in real-world applications, ranging from social networks to biological systems, and have inspired the development of Graph Neural Networks (GNNs) for learning expressive representations. While most research has centered on static graphs, many real-world scenarios involve dynamic, temporally evolving graphs, motivating the need for Continuous-Time Dynamic Graph (CTDG) models. This paper provides a comprehensive review of Graph Representation Learning (GRL) on CTDGs with a focus on Self-Supervised Representation Learning (SSRL). We introduce a novel theoretical framework that analyzes the expressivity of CTDG models through an Information-Flow (IF) lens, quantifying their ability to propagate and encode temporal and structural information. Leveraging this framework, we categorize existing CTDG methods based on their suitability for different graph types and application scenarios. Within the same scope, we examine the design of SSRL methods tailored to CTDGs, such as predictive and contrastive approaches, highlighting their potential to mitigate the reliance on labeled data. Empirical evaluations on synthetic and real-world datasets validate our theoretical insights, demonstrating the strengths and limitations of various methods across long-range, bi-partite and community-based graphs. This work offers both a theoretical foundation and practical guidance for selecting and developing CTDG models, advancing the understanding of GRL in dynamic settings.

Expressivity of Representation Learning on Continuous-Time Dynamic Graphs: An Information-Flow Centric Review

TL;DR

The paper addresses expressivity in continuous-time dynamic graphs by introducing an information-flow framework that quantifies how events update node representations through memory, temporal projection, and MP components. It categorizes CTDG methods into non-MP, MP-based, and hybrid families, linking architectural choices to their theoretical information propagation bounds, and ties SSRL methods to the same expressivity lens. Empirical validation across synthetic and real datasets demonstrates that MP-based models, especially CTAN and TGN variants, effectively capture long-range and community-structural dynamics, while SSRL pretext tasks benefit from methods with strong propagation capabilities. The work provides a principled roadmap for selecting and designing CTDG models with desired expressivity and data efficiency for diverse graph types and downstream tasks.

Abstract

Graphs are ubiquitous in real-world applications, ranging from social networks to biological systems, and have inspired the development of Graph Neural Networks (GNNs) for learning expressive representations. While most research has centered on static graphs, many real-world scenarios involve dynamic, temporally evolving graphs, motivating the need for Continuous-Time Dynamic Graph (CTDG) models. This paper provides a comprehensive review of Graph Representation Learning (GRL) on CTDGs with a focus on Self-Supervised Representation Learning (SSRL). We introduce a novel theoretical framework that analyzes the expressivity of CTDG models through an Information-Flow (IF) lens, quantifying their ability to propagate and encode temporal and structural information. Leveraging this framework, we categorize existing CTDG methods based on their suitability for different graph types and application scenarios. Within the same scope, we examine the design of SSRL methods tailored to CTDGs, such as predictive and contrastive approaches, highlighting their potential to mitigate the reliance on labeled data. Empirical evaluations on synthetic and real-world datasets validate our theoretical insights, demonstrating the strengths and limitations of various methods across long-range, bi-partite and community-based graphs. This work offers both a theoretical foundation and practical guidance for selecting and developing CTDG models, advancing the understanding of GRL in dynamic settings.

Paper Structure

This paper contains 31 sections, 4 theorems, 18 equations, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Theoretical Expressivity Framework
  3. Key Components of CTDG Representation Learning
  4. Expressivity Framework Based on Information Flow (IF)
  5. Method Categorization and Review
  6. Method Categorization Inspired by Theoretical Framework
  7. Non-message-passing (Non-MP) methods
  8. Message-passing (MP) based methods
  9. Hybrid methods
  10. Relevance of Self-Supervised Representation Learning (SSRL) Methods for CTDG
  11. Pretext tasks for pretraining on CTDG
  12. Downstream tasks
  13. Empirical Validation
  14. Experimental Setup
  15. Results Analysis
  16. ...and 16 more sections

Key Result

Theorem 2.2

Let's consider a CTDG-based function $f: (\mathcal{A}, \mathcal{X}) \rightarrow \mathcal{Y}$ based on $L$ GCN-like layers. After an event between node $i$ and another node, the following properties hold for any node $u$ not involved in the event:

Figures (3)

  • Figure 1: Categorization and taxonomy of GRL methods on CTDGs.
  • Figure 2: Difference between the theoretical upper-bound and the empirical values of the difference in norms for a node $u$'s representation.
  • Figure 3: Information Flow change after an event occurring and the Average Test MRR on the TGBL-Wiki dataset for different number of layers.

Theorems & Definitions (7)

  • Definition 2.1: Continuous IF
  • Theorem 2.2: GCN-based aggregation
  • Theorem 2.3: Attention-based aggregation
  • Theorem : GCN-based aggregation
  • proof
  • Theorem : Attention-based aggregation
  • proof