Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Information propagation dynamics in Deep Graph Networks

Alessio Gravina

TL;DR

This thesis investigates the dynamics of information propagation within DGNs for static and dynamic graphs, focusing on their design as dynamical systems, offering insights and advancements for the field of graph representation learning and paving the way for more effective and versatile graph-based learning models.

Abstract

Graphs are a highly expressive abstraction for modeling entities and their relations, such as molecular structures, social networks, and traffic networks. Deep Graph Networks (DGNs) have emerged as a family of deep learning models that can effectively process and learn such structured information. However, learning effective information propagation patterns within DGNs remains a critical challenge that heavily influences the model capabilities, both in the static domain and in the temporal domain (where features and/or topology evolve). Given this challenge, this thesis investigates the dynamics of information propagation within DGNs for static and dynamic graphs, focusing on their design as dynamical systems. Throughout this work, we provide theoretical and empirical evidence to demonstrate the effectiveness of our proposed architectures in propagating and preserving long-term dependencies between nodes, and in learning complex spatio-temporal patterns from irregular and sparsely sampled dynamic graphs. In summary, this thesis provides a comprehensive exploration of the intersection between graphs, deep learning, and dynamical systems, offering insights and advancements for the field of graph representation learning and paving the way for more effective and versatile graph-based learning models.

Information propagation dynamics in Deep Graph Networks

TL;DR

This thesis investigates the dynamics of information propagation within DGNs for static and dynamic graphs, focusing on their design as dynamical systems, offering insights and advancements for the field of graph representation learning and paving the way for more effective and versatile graph-based learning models.

Abstract

Graphs are a highly expressive abstraction for modeling entities and their relations, such as molecular structures, social networks, and traffic networks. Deep Graph Networks (DGNs) have emerged as a family of deep learning models that can effectively process and learn such structured information. However, learning effective information propagation patterns within DGNs remains a critical challenge that heavily influences the model capabilities, both in the static domain and in the temporal domain (where features and/or topology evolve). Given this challenge, this thesis investigates the dynamics of information propagation within DGNs for static and dynamic graphs, focusing on their design as dynamical systems. Throughout this work, we provide theoretical and empirical evidence to demonstrate the effectiveness of our proposed architectures in propagating and preserving long-term dependencies between nodes, and in learning complex spatio-temporal patterns from irregular and sparsely sampled dynamic graphs. In summary, this thesis provides a comprehensive exploration of the intersection between graphs, deep learning, and dynamical systems, offering insights and advancements for the field of graph representation learning and paving the way for more effective and versatile graph-based learning models.

Paper Structure

This paper contains 126 sections, 14 theorems, 189 equations, 42 figures, 53 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Objectives
  3. Thesis Outline
  4. Origin of the Chapters
  5. Background
  6. Preliminaries
  7. Graphs
  8. Static Graphs
  9. Dynamic Graphs
  10. Introduction to Differential Equations
  11. Differential Equations
  12. Hamiltonian Systems
  13. Discretization Methods
  14. Neural Differential Equations
  15. Fundamentals of DGNs
  16. ...and 111 more sections

Key Result

Proposition 1

Assuming that $\mathbf{J}(t)$ does not change significantly over time, the forward and backward propagations of the ODE in Equation eq:simple_ode_dgn are stable and non-dissipative if

Figures (42)

  • Figure 1: (a) A directed graph. (b) An undirected graph. (c) The neighborhood of node $u$. At the bottom, the adjacency matrix of the corresponding graph. The neighborhood of the node $u$ is depicted with a dotted line.
  • Figure 2: Instances of various graph families: (a) path graph, (b) ring graph, (c) crossed-ring graph, (d) grid graph, (e) Erdős–Rényi Graph with probability $p=0.2$, and (f) Barabasi-Albert Graph with $k=2$. Each graph consists of 10 nodes.
  • Figure 3: A Discrete-Time Dynamic Graph defined over five timestamps and a set of five interacting entities.
  • Figure 4: The evolution of a Continuous-Time Dynamic Graph through the stream of events until the timestamp $t_3$.
  • Figure 5: Motion of a pendulum.
  • ...and 37 more figures

Theorems & Definitions (60)

  • Definition 1: Graph
  • Definition 2: Directed/Undirected graph
  • Definition 3: Neighborhood
  • Definition 4: In-degree/Out-degree
  • Definition 5: Graph Laplacian
  • Definition 6: Symmetric normalized graph Laplacian
  • Definition 7: Random-Walk normalized graph Laplacian
  • Definition 8: Path graph
  • Definition 9: Ring graph
  • Definition 10: Crossed-ring graph
  • ...and 50 more