Symbolic Neural Ordinary Differential Equations

TL;DR

SNODEs address learning the operator from the parametric input function $\mathbf{u}(\mathbf{x},t)$ to the system state $\mathbf{s}(\mathbf{x},t)$ in parametric dynamical systems. They couple symbolic regression via SymNet with Neural ODEs and a GeNN residual learner in a three-stage training pipeline, using Fourier-based spatial derivative handling to achieve resolution-invariant PDE modeling. A universal approximation theorem for SNODEs is established, and experiments on 1-D ODEs, bifurcations, and parametric PDEs show improved interpretability and extrapolation over baselines. The framework enables accurate, interpretable, and extrapolatable discovery and forecasting for complex spatiotemporal dynamics, with broad relevance to system reconstruction, forecasting, and equation discovery.

Abstract

Differential equations are widely used to describe complex dynamical systems with evolving parameters in nature and engineering. Effectively learning a family of maps from the parameter function to the system dynamics is of great significance. In this study, we propose a novel learning framework of symbolic continuous-depth neural networks, termed Symbolic Neural Ordinary Differential Equations (SNODEs), to effectively and accurately learn the underlying dynamics of complex systems. Specifically, our learning framework comprises three stages: initially, pre-training a predefined symbolic neural network via a gradient flow matching strategy; subsequently, fine-tuning this network using Neural ODEs; and finally, constructing a general neural network to capture residuals. In this process, we apply the SNODEs framework to partial differential equation systems through Fourier analysis, achieving resolution-invariant modeling. Moreover, this framework integrates the strengths of symbolism and connectionism, boasting a universal approximation theorem while significantly enhancing interpretability and extrapolation capabilities relative to state-of-the-art baseline methods. We demonstrate this through experiments on several representative complex systems. Therefore, our framework can be further applied to a wide range of scientific problems, such as system bifurcation and control, reconstruction and forecasting, as well as the discovery of new equations.

Figures (5)

• Figure 1: The sketched framework of SNODEs. This framework, which includes the SymNet and GeNN components, takes state variables, parametric functions, and all possible spatial partial derivatives as inputs. It models a family of parametric dynamical systems using the proposed three-stage and adaptive learning strategy.
• Figure 2: Operator learning for system \ref{['ode']}. (a) The mean squared error (MSE) of different methods and different $N_\mathrm{tr}$ in the testing set. (b) The MSE in the extrapolation experiment. Here, $m$ in "DON($m$)" represents the number of the uniform sampling points of the parameter function.
• Figure 3: Experimental results in systems with saddle-node, pitchfork and Hopf bifurcations. (a), (c), (e) Predicting bifurcation dynamics using SNODEs. (b), (d) Predicting bifurcation dynamics using DeepONet. Here, the green dashed lines indicate the training data, the red dots represent the true bifurcation diagram, the arrows depict the predicted vector field, the black line represents the true trajectory, and the red line represent predicted trajectory.
• Figure 4: Predicting DR and KS systems using SNODEs. (a) and (b) show the testing examples for the DR and KS systems, respectively. Here, the training set features a spatial resolution of $N_x = 32$, whereas the test set has $N_x = 128$. (c), (d), and (e) respectively illustrate the variations in training and validation losses across three training stages.
• Figure 5: Predicting NS systems using SNODEs. (a) The predicted result for training data with spatial resolution $N_x=N_y=16$. (b) The predicted result for testing data with spatial resolution $N_x=N_y=80$.

Theorems & Definitions (2)

• Theorem 1
• Theorem 2