Learning Interpretable Network Dynamics via Universal Neural Symbolic Regression

TL;DR

A universal computational tool that combines deep learning with symbolic regression to automatically and efficiently uncover the underlying equations driving complex systems and can serve as a universal solution to dispel the fog of hidden mechanisms of changes in complex phenomena.

Abstract

Discovering governing equations of complex network dynamics is a fundamental challenge in contemporary science with rich data, which can uncover the mysterious patterns and mechanisms of the formation and evolution of complex phenomena in various fields and assist in decision-making. In this work, we develop a universal computational tool that can automatically, efficiently, and accurately learn the symbolic changing patterns of complex system states by combining the excellent fitting ability from deep learning and the equation inference ability from pre-trained symbolic regression. We conduct intensive experimental verifications on more than ten representative scenarios from physics, biochemistry, ecology, epidemiology, etc. Results demonstrate the outstanding effectiveness and efficiency of our tool by comparing with the state-of-the-art symbolic regression techniques for network dynamics. The application to real-world systems including global epidemic transmission and pedestrian movements has verified its practical applicability. We believe that our tool can serve as a universal solution to dispel the fog of hidden mechanisms of changes in complex phenomena, advance toward interpretability, and inspire more scientific discoveries.

Figures (15)

• Figure 1: The overall process of the LLC (Learning Law of Changes). Observed data can be acquired from the initial experiments on the new scenario, including system states over time and topology, i.e. $O$. An interval selection strategy is to choose valid interval data and then we can get differentials, i.e. $\dot{X}_i(t)$, through finite difference on ${X}_i(t)$. Combined with physical priors, the neural networks, i.e., $\Hat{Q}_{\theta_1}^{(self)}$ and $\Hat{Q}_{\theta_2}^{(inter)}$, are to decouple the network dynamics signals and achieve variable reduction, until learning to meet the fitting requirements. Otherwise, the training is repeated. After having well-fitted neural networks, we use symbolic regression techniques to parse their approximate white-box equations efficiently. Of course, suppose more observed data is needed to support the discovery. In that case, the experimental design can be revisited until the satisfactory governing equations of network dynamics are finally acquired to break out of the loop.
• Figure 2: Results of inferring one-dimensional homogeneous network dynamics. a. Comparison of the accuracy on predictions ($R^2$ score) and discovered equations (Recall) for reconstructing dynamics from six scenarios, including Biochemical (Bio), Gene regulatory (Gene), Mutualistic Interaction (MI), Lotka-Volterra (LV), Neural (Neur), and Epidemic (Epi) dynamics. TPSINDy's results are highly dominated by its choice of function terms and our LLC significantly outperforms the comparative methods covering all network dynamics scenarios. b. Comparison of the average execution times across all dynamics for various symbolic regression methods. c. The NED (Normalized Estimation Error) between the predictive results produced by the discovered governing equations and ground truth in the LV scenario. d. Comparison of fitting coefficients in governing equations discovered by symbolic regression methods. e. Comparison of state prediction curves for an individual node.
• Figure 3: Results of inferring the FitzHugh-Nagum (FHN) dynamics. a. The fitting results of the first dimension ($\dot{X}_{i,1}(t)$) on a node by neural networks. b. The decoupling results of the self dynamics for the first dimension on a node ($\Hat{Q}_{\theta_1}^{(self)}$). c. The decoupling results of the interaction dynamics for the first dimension on a node ($\Hat{Q}_{\theta_2}^{(inter)}$). d. The fitting results of the second dimension ($\dot{X}_{i,2}(t)$) on a node by neural networks. e. Comparison of governing equations inferred by various methods. f. Comparison of the normalized Euclidean distance (NED) between two trajectories generated separately from the inferred and true equations, where the horizontal axis represents the node index. g. Comparison of trajectories generated by the inferred and true equations on a Barabási-Albert network.
• Figure 4: Results of inferring the predator-prey (PP) system. a. Comparison of governing equations inferred by various methods. b. The ground truth positions of a predator (square) and prey swarm (dots) over time. c. The predictive positions generated by the governing equation inferred by the TPSINDy. d. The predictive positions generated by the governing equation inferred by the LCC.
• Figure 5: Results of inferring the dynamics of chaotic systems. a. Comparison of governing equations inferred by our LCC under different initial conditions on the coupled Lorentz system. b. Comparison of predictive trajectories with the same initial values, produced by equations inferred by our LLC under different initial conditions. c. Coefficient errors between the equations inferred by each method on the Rossler system and the true equation. d. Comparison of trajectories on the same node, generated by the governing equations inferred by the TPSINDy, LCC, and LCC+TPSINDy on the Rossler system. e. Bifurcation diagram of the Rossler system via the Poincaré section method, with the horizontal axis representing the parameter $c$ (ranging from 1 to 6) and the vertical axis representing the system’s state on the second dimension ($X_{i,2}$) of a node. The discovered equation exhibits the same period-doubling and chaotic phenomenon as the true equation. f. Comparison of limit cycle at period-1, i.e., $c=2.5$. g. Comparison of chaos at $c=5.7$.