Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Dynamic Graph Embeddings with Neural Controlled Differential Equations

Tiexin Qin, Benjamin Walker, Terry Lyons, Hong Yan, Haoliang Li

TL;DR

This work tackles learning representations on continuous-time dynamic graphs where both topology and node dynamics evolve. It introduces Graph Neural Controlled Differential Equations (GN-CDEs), which drive node embeddings with a graph-aware neural differential equation controlled by a time-varying graph path $\\hat{\\mathbf{A}}$, enabling seamless, calibrated updates as new data arrive. The authors establish theoretical guarantees (existence/uniqueness), propose scalable approximations, and demonstrate strong empirical performance on dynamic node attribute prediction and temporal link prediction, outperforming several neural ODE/CDE baselines and many temporal GNNs. The framework is flexible, extends to various graph types, and offers memory-efficient training, making it well-suited for practical dynamic graph tasks with irregular sampling.

Abstract

This paper focuses on representation learning for dynamic graphs with temporal interactions. A fundamental issue is that both the graph structure and the nodes own their own dynamics, and their blending induces intractable complexity in the temporal evolution over graphs. Drawing inspiration from the recent progress of physical dynamic models in deep neural networks, we propose Graph Neural Controlled Differential Equations (GN-CDEs), a continuous-time framework that jointly models node embeddings and structural dynamics by incorporating a graph enhanced neural network vector field with a time-varying graph path as the control signal. Our framework exhibits several desirable characteristics, including the ability to express dynamics on evolving graphs without piecewise integration, the capability to calibrate trajectories with subsequent data, and robustness to missing observations. Empirical evaluation on a range of dynamic graph representation learning tasks demonstrates the effectiveness of our proposed approach in capturing the complex dynamics of dynamic graphs.

Learning Dynamic Graph Embeddings with Neural Controlled Differential Equations

TL;DR

This work tackles learning representations on continuous-time dynamic graphs where both topology and node dynamics evolve. It introduces Graph Neural Controlled Differential Equations (GN-CDEs), which drive node embeddings with a graph-aware neural differential equation controlled by a time-varying graph path , enabling seamless, calibrated updates as new data arrive. The authors establish theoretical guarantees (existence/uniqueness), propose scalable approximations, and demonstrate strong empirical performance on dynamic node attribute prediction and temporal link prediction, outperforming several neural ODE/CDE baselines and many temporal GNNs. The framework is flexible, extends to various graph types, and offers memory-efficient training, making it well-suited for practical dynamic graph tasks with irregular sampling.

Abstract

This paper focuses on representation learning for dynamic graphs with temporal interactions. A fundamental issue is that both the graph structure and the nodes own their own dynamics, and their blending induces intractable complexity in the temporal evolution over graphs. Drawing inspiration from the recent progress of physical dynamic models in deep neural networks, we propose Graph Neural Controlled Differential Equations (GN-CDEs), a continuous-time framework that jointly models node embeddings and structural dynamics by incorporating a graph enhanced neural network vector field with a time-varying graph path as the control signal. Our framework exhibits several desirable characteristics, including the ability to express dynamics on evolving graphs without piecewise integration, the capability to calibrate trajectories with subsequent data, and robustness to missing observations. Empirical evaluation on a range of dynamic graph representation learning tasks demonstrates the effectiveness of our proposed approach in capturing the complex dynamics of dynamic graphs.
Paper Structure (30 sections, 4 theorems, 37 equations, 5 figures, 4 tables, 1 algorithm)

This paper contains 30 sections, 4 theorems, 37 equations, 5 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Introduction
  3. Related Works
  4. Paper organization and notations
  5. Preliminary
  6. Graph Embedding Learning
  7. Neural Differential Equations
  8. Methodology
  9. Problem Setup
  10. GN-CDEs: Graph Neural Controlled Differential Equations
  11. Properties
  12. Comparison to Alternative Models
  13. Approximation of GN-CDE
  14. Applications
  15. Experiments
  16. ...and 15 more sections

Key Result

Theorem 2

If $f_{\theta}$ is globally Lipschitz continuous and $\hat{\mathbf{A}}_t$ is a piecewise continuously differentiable interpolation, then the solution $\mathbf{Z}_t$ of eq:gn_cde exhibits global existence and uniqueness.

Figures (5)

  • Figure 1: An overview of our proposed GN-CDE. In the canonical setting, dynamic graphs are characterized by an evolving adjacency matrix but can be extended to more sophisticated scenarios as discussed in Section \ref{['sec:method_expansion']}.
  • Figure 2: Heat Diffusion: The test errors of Neural ODE, Neural CDE and our GN-CDE models with respect to the optimization iteration count under five different graph structures: (a) grid, (b) Random, (c) power law, (d) small world and (e) community.
  • Figure 3: Gene Regulation: The test errors of Neural ODE, Neural CDE and our GN-CDE models with respect to the optimization iteration count under five different graph structures: (a) grid, (b) Random, (c) power law, (d) small world and (e) community.
  • Figure 4: Visualization of learned dynamics for heat diffusion over dynamic graphs. Our GN-CDE model fits the dynamics for the whole progress accurately. All experiments have been averaged over 10 random seeds with the stand deviation reported.
  • Figure 5: Illustration of other graph types.

Theorems & Definitions (9)

  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Theorem 4
  • proof
  • Theorem 5
  • proof
  • Definition 6: Maximal expressivity walker2025structured