Neural Graph Simulator for Complex Systems

TL;DR

The Neural Graph Simulator (NGS) is introduced for simulating time-invariant autonomous systems on graphs, providing a unified framework to simulate diverse dynamical systems with varying topologies and sizes without constraints on evaluation times through its non-uniform time step and autoregressive approach.

Abstract

Numerical simulation is a predominant tool for studying the dynamics in complex systems, but large-scale simulations are often intractable due to computational limitations. Here, we introduce the Neural Graph Simulator (NGS) for simulating time-invariant autonomous systems on graphs. Utilizing a graph neural network, the NGS provides a unified framework to simulate diverse dynamical systems with varying topologies and sizes without constraints on evaluation times through its non-uniform time step and autoregressive approach. The NGS offers significant advantages over numerical solvers by not requiring prior knowledge of governing equations and effectively handling noisy or missing data with a robust training scheme. It demonstrates superior computational efficiency over conventional methods, improving performance by over $10^5$ times in stiff problems. Furthermore, it is applied to real traffic data, forecasting traffic flow with state-of-the-art accuracy. The versatility of the NGS extends beyond the presented cases, offering numerous potential avenues for enhancement.

Figures (7)

• Figure 1: (a) Schematic diagram of the structure of the NGS. The state $\bm{S}(t_m)$, time step $\Delta t_m$, adjacency matrix, and coefficients $\bm{V}, \bm{E}, \bm{g}$ are provided as input. The corresponding encoders lift the vectors into high-dimensional latent vectors $\bm{V}^{(0)}, \bm{E}^{(0)}, \bm{g}^{(0)}$. Subsequently, the latent vectors undergo $L$ layers of Graph Network (GN), after which they are decoded back to the physical dimensions. (b) Computation of the $l$-th GN layer. The main text provides a detailed description. (c) Random state of the thermal system, where node colors indicate the temperature. (d) Degradation of the data. Gaussian noise is represented using color, and two missing nodes are marked with black squares. (e) Incomplete data used to train the NGS. For a model with a depth of 2, the second-nearest neighbor of the missing nodes (marked with squares) does not participate in the training.
• Figure 2: (a), (b) Snapshots of the thermal system at three-time points simulated by the NGS and DOP853, respectively. Node color indicates temperature, and the edge thickness represents the dissipation rate. Square nodes represent those marked as missing during training. (c) Discrepancies in temperatures of all nodes between the NGS and DOP853 at non-uniform evaluation times. The three evaluation times used in (a) and (b) are marked by red squares. The box represents the first and third quantile, with whiskers extending to 1.5 times the inter-quantile range. (d), (e) NFEV and runtime of the NGS and DOP853 on a logarithmic scale. Efficiencies are illustrated in orange for the NGS and purple for the numerical solver. Results are depicted seperately for the two graph domains $\mathcal{G}_\text{int}$ and $\mathcal{G}_\text{ext}$, on time interval $\mathcal{T}_\text{int} \cup \mathcal{T}_\text{ext}$.
• Figure 3: (a) Trajectories of $x, y$, and $z$ coordinate for three nodes in the coupled Rössler system. Each node is color-coded differently (red, green, and blue). The DOP853 simulation is represented by a solid line, while states simulated by the NGS and DOP853 at non-uniform time points are marked by crosses and circles, respectively. (b) Trajectory of the same nodes as shown in (a), presented in three-dimensional space. (c) Lyapunov exponents in the NGS and DOP853 simulations, with discrepancies in the initial condition following a normal distribution with different standard deviations. (d), (e) NFEV and runtime of the NGS and DOP853 on a logarithmic scale, respectively. The efficiency of the NGS is illustrated in orange, while that of the numerical solver is depicted in purple. Results for two graph domains $\mathcal{G}_\text{int}$ and $\mathcal{G}_\text{ext}$ are presented, on time interval $\mathcal{T}_\text{int} \cup \mathcal{T}_\text{ext}$.
• Figure 4: (a) Precise snapshots of the Kuramoto system simulated by LSODA with $\theta_\text{th}=\pi/2$. The position and color of the nodes at each evaluation time represent the phase of the corresponding oscillator. With $\theta_\text{th}=\pi/2$, all interactions are encompassed using a fully connected graph. (b), (c) Snapshots of the Kuramoto system shown in (a) simulated by the NGS and LSODA with $\theta_\text{th}=\pi/6$, respectively. The positions of nodes represent the phases predicted by each simulator, while the colors represent the errors compared to (a). An edge is established between nodes only when their phase difference is close to $\pi/2$ or $3\pi/2$. (d), (e) Errors at all evaluation times in the NGS and LSODA with $\theta_\text{th}=\pi/6$. The colors in each row represent the error for each oscillator, employing the same color scheme as in (b) and (c). Black solid lines indicate non-uniform evaluation times, with the three points used in (a), (b), and (c) marked with blue squares. (f) The mean absolute error (MAE) for the NGS and LSODA using four different values of $\theta_\text{th}$ on a logarithmic scale. (g), (h) NFEV and runtime of NGS and LSODA simulations with four different $\theta_\text{th}$ on a logarithmic scale. For LSODA, the NFEV for $\theta_\text{th} < \pi/2$ is more than $10^5$ times greater than that for $\theta_\text{th} = \pi/2$, indicating stiffness.
• Figure S1: Mean absolute error (MAE) with the unit of $10^{-4}$, trained with various noise levels $\sigma$ and missing fractions $p$ in the thermal system. The color of each cell represents the average MAE in each domain on a logarithmic scale. The black square represents the $\sigma=0.001, p=0.1$ case, discussed in the main text.