Graph Neural Ordinary Differential Equations

TL;DR

Graph Neural Ordinary Differential Equations (GDEs) introduce a continuous-depth extension of graph neural networks by formulating inter-layer propagation as an ODE on graphs, solved over a horizon with a depth-varying vector field $\dot{H}(s)=F_{\mathcal{G}}(s,H(s),\Theta)$. The framework unifies static and spatio-temporal graphs, supports various solvers and discretizations, and includes a rigorous treatment of well-posedness, integration domains, and training strategies (including adjoint methods). Empirical evaluations across node classification, multi-agent trajectory extrapolation, and traffic forecasting show GCDEs often outperform their discrete counterparts, with higher-order solvers and stable performance under varying integration depth; autoregressive GDEs further enable handling irregular timestamps. Overall, GDEs offer a principled, versatile approach to embedding dynamical-system perspectives into GNNs, enabling robust modeling of relational data with continuous dynamics and potential computational advantages in static tasks through solver-based forward passes.

Abstract

We introduce the framework of continuous--depth graph neural networks (GNNs). Graph neural ordinary differential equations (GDEs) are formalized as the counterpart to GNNs where the input-output relationship is determined by a continuum of GNN layers, blending discrete topological structures and differential equations. The proposed framework is shown to be compatible with various static and autoregressive GNN models. Results prove general effectiveness of GDEs: in static settings they offer computational advantages by incorporating numerical methods in their forward pass; in dynamic settings, on the other hand, they are shown to improve performance by exploiting the geometry of the underlying dynamics.

Figures (12)

• Figure 1: Graph neural ordinary differential equations (GDEs) model vector fields defined on graphs, both in cases when the structure is fixed or changes in time, via a continuum of graph neural network (GNN) layers.
• Figure 2: Schematic of autoregressive GDEs as hybrid automata.
• Figure 3: Node embedding trajectories defined by a forward pass of GCDE--dpr5 on Cora, Citeseer and Pubmed. Color differentiates between node classes.
• Figure 4: Cora accuracy of GCDE models with different integration times $s$.
• Figure 5: Example position and velocity trajectories of the multi--particle system.