Permutation Equivariant Neural Controlled Differential Equations for Dynamic Graph Representation Learning

TL;DR

This work introduces Permutation Equivariant Neural Graph Controlled Differential Equations (PENG-CDEs) for dynamic graph representation learning. By projecting the fusion of adjacency and its time derivative onto the permutation-equivariant subspace $\mathfrak{E}_{\Sigma_n}(2,1)^1$, it achieves parameter efficiency while preserving expressivity, and ensures both permutation and time-warp equivariance. Theoretical results establish optimality of the projection in the linear case, and the framework extends to dynamic node features with a node-wise Hadamard interaction, maintaining symmetry. Empirically, PENG-CDEs outperform non-equivariant and several baselines on synthetic heat diffusion and gene regulation tasks, real-world snapshots, and TGB node-affinity benchmarks, while also exhibiting robustness to oversampling and irregular time grids. Limitations include quadratic memory growth with node count due to dense adjacency storage, with future work aiming to scalable sparse representations, advanced solvers, and potential theoretical guarantees.

Abstract

Dynamic graphs exhibit complex temporal dynamics due to the interplay between evolving node features and changing network structures. Recently, Graph Neural Controlled Differential Equations (Graph Neural CDEs) successfully adapted Neural CDEs from paths on Euclidean domains to paths on graph domains. Building on this foundation, we introduce Permutation Equivariant Neural Graph CDEs, which project Graph Neural CDEs onto permutation equivariant function spaces. This significantly reduces the model's parameter count without compromising representational power, resulting in more efficient training and improved generalisation. We empirically demonstrate the advantages of our approach through experiments on simulated dynamical systems and real-world tasks, showing improved performance in both interpolation and extrapolation scenarios.

Figures (5)

• Figure 1: Test losses, plotted against simulation time, for the Graph Neural ODE, three GN-CDE variants, and our proposed Permutation-Equivariant GN-CDE model on the heat diffusion (left) and gene regulation (right) tasks. Dashed vertical lines mark changes in graph topology, while the bold black line indicates the final time point in the training set. Results are reported as means (solid) and ranges (shaded) over a test set with a batch size of four.
• Figure 2: Classification accuracy on the SIR model for STIDGCN, ASTGCN, DCRNN, and our PENG-CDE as a function of the number of observed timesteps (left) and sampling irregularity (right). Inference time with increasing numbers of observations is shown in the middle panel.
• Figure 3: Critical difference diagrams for the synthetic experiments in Section \ref{['sec:exp_synth']} and Tables \ref{['tab:combined']} and \ref{['tab:combined_additional']}.
• Figure 4: Critical difference diagrams for the Pytorch Geometric Temporal (PGT) real-world experiments in Section \ref{['sec:exp_real_world']} and Table \ref{['tab:pgt_erperiments']}.
• Figure 5: Critical difference diagrams for the Temporal Graph Benchmark (TGB) real-world experiments in Section \ref{['sec:exp_real_world']} and Table \ref{['tab:tgb']}.

Theorems & Definitions (26)

• Theorem 3.1
• proof : Proof sketch
• Proposition 3.2
• proof
• Definition B.1: Projection
• Theorem B.2: Projection Theorem, e.g. aubin1979applied, Chapter $1$
• Definition C.1: Group Action
• Definition C.2: Group Representation
• Definition C.3: Equivariance
• Definition C.4: Haar measure