Learning Continuous Network Emerging Dynamics from Scarce Observations via Data-Adaptive Stochastic Processes

TL;DR

Intensive experiments conducted on various network dynamics in ecological population evolution, phototaxis movement, brain activity, epidemic spreading, and real-world empirical systems, demonstrate that the proposed method has excellent data adaptability and computational efficiency, and can adapt to unseen network emerging dynamics, producing accurate interpolation and extrapolation with reducing the ratio of required observation data and improving the learning speed for new dynamics by three orders of magnitude.

Abstract

Learning network dynamics from the empirical structure and spatio-temporal observation data is crucial to revealing the interaction mechanisms of complex networks in a wide range of domains. However, most existing methods only aim at learning network dynamic behaviors generated by a specific ordinary differential equation instance, resulting in ineffectiveness for new ones, and generally require dense observations. The observed data, especially from network emerging dynamics, are usually difficult to obtain, which brings trouble to model learning. Therefore, how to learn accurate network dynamics with sparse, irregularly-sampled, partial, and noisy observations remains a fundamental challenge. We introduce Neural ODE Processes for Network Dynamics (NDP4ND), a new class of stochastic processes governed by stochastic data-adaptive network dynamics, to overcome the challenge and learn continuous network dynamics from scarce observations. Intensive experiments conducted on various network dynamics in ecological population evolution, phototaxis movement, brain activity, epidemic spreading, and real-world empirical systems, demonstrate that the proposed method has excellent data adaptability and computational efficiency, and can adapt to unseen network emerging dynamics, producing accurate interpolation and extrapolation with reducing the ratio of required observation data to only about 6\% and improving the learning speed for new dynamics by three orders of magnitude.

Figures (8)

• Figure 1: The overall workflow of the NDP4ND.a, Empirical data from network dynamics, including network topology $A$ and state observations $\mathcal{D}_{\mathbb{C}}$. b, Generating the distribution over the global random vector $z$, i.e., $p(z|\mathcal{D}_{\mathbb{C}},A)$ by encoding temporal and topological information. c, Generating the distribution over the initial states in latent space, i.e., $p(L(0)|\mathcal{D}_{\mathbb{C}},A)$. d, After obtaining the distributions of $z$ and $L(0)$, we perform dynamic propagation in the hidden space based on the universal skeleton of network dynamics and give the differentiation of hidden states on node $l$ at time $t$, i.e., $\dot{L}_{l}(t)$. e, Calculating the hidden states ${L}_{l}(t)$ and then output the state distribution on any node $l$ at any time $t$, i.e., $p(X_l(t)|t,l,z,L(0),A)$. Note that neural networks ($\bm{\varphi}$, $\bm{\rho}$, $\bm{e}$, $\bm{S}$, $\bm{I}$, $\bm{d}$) are to parameterize the stochastic processes and achieve adaptive learning from data.
• Figure 1: The interpolation and extrapolation results on susceptible-infectious-recovered (SIR) and susceptible-exposed-infectious-susceptible (SEIS). a, The testing results of the NDCN, DNND, and our NDP4ND on the SIR. The ratio of observations in this testing network ODE is $6.89\%$. b, The testing results of the NDCN, DNND, and our NDP4ND on the SEIS. The ratio of observations in this testing network ODE is $4.08\%$. The number of nodes and the maximum value of all observed times ($T_o$) are 200 and 50, respectively. Overall, our NDP4ND can learn the most effective network dynamics from irregularly-sampled, partial, and sparse observations.
• Figure 2: The interpolation and extrapolation results on mutualistic interaction dynamics. a-d, The testing results of the best-performing and worst-performing three nodes of the LG-ODE, NDCN, DNND, and our NDP4ND. The ratio of observations in this testing network ODE is $3.06\%$. The number of nodes and the maximum value of all observed times ($T_o$) are 225 and 50, respectively. Our NDP4ND has the best performance in both interpolation and extrapolation, and ultimately stabilizes at two population values. e, The distributions of Mean Absolute Error (MAE) between predictions and ground truth for all nodes, demonstrating the high-precision predictions from our method. f-h, Our NDP4ND can directly utilize newly arrived data and does not require any retraining, with adaptively rectifying predictions toward ground truth as observations increase. Note that the subfigures only show the observation data on six nodes instead of all nodes. i, The average learning time for all testing network ODEs, showing that our NDP4ND improves the learning speed for new dynamics by three orders of magnitude.
• Figure 2: a, The state changes on node $l$ can be modeled by self dynamics and interaction dynamics, i.e., $\frac{dX_{l}(t)}{dt}=f(X_l(t))+\sum_{j}{A_{l,j}g(X_l(t),X_j(t))}$. The former quantifies the impact of one's own state on change, while the latter quantifies the impact of neighboring states on change. b, We model a collection of $\{X_l(t)\}$ as a stochastic process, where the state $X_l(t)$ on node $l$ at time $t$ is a random variable. In principle, as time and space (network) expand, there are infinitely many such random variables in a networked system. c, The probabilistic graphical model of the generative process behind the NDP4ND. $z$ and $L(0)$ denote the global random vector and the initial states in latent space respectively, through which uncertainty is introduced to enhance the model prediction facing scarce and noisy observations.
• Figure 3: The interpolation and extrapolation results on phototaxis dynamics. a-d, The ground truth and testing results of the NDCN, DNND, and our NDP4ND. The ratio of observations in this testing network ODE is $3.23\%$. The number of nodes and the maximum value of all observed times ($T_o$) are 40 and 0.5, respectively. The predictive five states, including coordinate 1, coordinate 2, velocity 1, velocity 2 and excitation level, are reported. The NDCN has not learned effective network dynamics from a few observations. The DNND fits well at the observed locations, but its velocity eventually converges to the wrong solution. Our NDP4ND ultimately stabilizes at the correct velocity and the learned dynamics are relatively stable and close to the ground truth. e-g, The distributions of Mean Absolute Error (MAE) between predictions and ground truth for all nodes, demonstrating the high-precision predictions from our method.