Temporal Graph ODEs for Irregularly-Sampled Time Series

TL;DR

The Temporal Graph Ordinary Differential Equation (TG-ODE) framework is introduced, which learns both the temporal and spatial dynamics from graph streams where the intervals between observations are not regularly spaced.

Abstract

Modern graph representation learning works mostly under the assumption of dealing with regularly sampled temporal graph snapshots, which is far from realistic, e.g., social networks and physical systems are characterized by continuous dynamics and sporadic observations. To address this limitation, we introduce the Temporal Graph Ordinary Differential Equation (TG-ODE) framework, which learns both the temporal and spatial dynamics from graph streams where the intervals between observations are not regularly spaced. We empirically validate the proposed approach on several graph benchmarks, showing that TG-ODE can achieve state-of-the-art performance in irregular graph stream tasks.

Figures (5)

• Figure 1: An example of a non-uniform sampling of a temporal graph with snapshots over a set of 5 nodes.
• Figure 2: The continuous processing of node $u$'s state in a discrete-time dynamic graph with irregularly-sampled snapshots over a set of 4 nodes and fixed edge set. At the top, the node-wise ODE function $f_\mathcal{\theta}$ defines the evolution of the states $\mathbf{x}_u(t)$. At the bottom, the discretized solution of the node-wise ODE, which corresponds to our framework TG-ODE. The node embedding $\mathbf{x}_u^\ell$ is computed iteratively over a discrete set of points by leveraging the temporal neighborhood and self-representation at the previous step.
• Figure 3: (a) A grid graph consisting of 70 nodes in which each node is characterized by an initial temperature. Darker colors correspond to colder temperatures, while brighter colors mean warmer temperatures. (b) The heat diffusion simulation is computed through 1000 steps forward Euler's method leveraging $-\mathbf{L}\mathbf{X}(t)$ as diffusion.
• Figure 4: Average time per epoch (measured in seconds) and std computed using an Intel Xeon Gold 6240R CPU @ 2.40GHz. Each time is obtained using 5 neighbor hops (when possible) and embedding dimension equal to 64. The graph size is computed as $size=\#steps * \#edges$.
• Figure 5: Test $\mathrm{MAE}$ scores and std of TG-ODE on PeMS04, averaged over 5 runs, for different sparsity levels.