An Empirical Investigation of Neural ODEs and Symbolic Regression for Dynamical Systems

TL;DR

This work evaluates how Neural Ordinary Differential Equations (NODEs) and Symbolic Regression (SR) perform in discovering and extrapolating the dynamics of damped oscillatory systems from noisy data. It shows that NODEs can extrapolate to unseen boundary conditions when new trajectories share the training dynamics, while SR can recover governing equations from ground-truth data, especially when the input $\lambda$ is included. Importantly, training a NODE on only 10% of the data to generate additional data for SR yields two of three governing equations and a good approximation for the third, illustrating a denoising effect and a data-efficient path to physical-law discovery. Together, the results propose a promising NODE-augmented SR pipeline for data-scarce scientific domains to infer underlying differential equations and physical laws.

Abstract

Accurately modelling the dynamics of complex systems and discovering their governing differential equations are critical tasks for accelerating scientific discovery. Using noisy, synthetic data from two damped oscillatory systems, we explore the extrapolation capabilities of Neural Ordinary Differential Equations (NODEs) and the ability of Symbolic Regression (SR) to recover the underlying equations. Our study yields three key insights. First, we demonstrate that NODEs can extrapolate effectively to new boundary conditions, provided the resulting trajectories share dynamic similarity with the training data. Second, SR successfully recovers the equations from noisy ground-truth data, though its performance is contingent on the correct selection of input variables. Finally, we find that SR recovers two out of the three governing equations, along with a good approximation for the third, when using data generated by a NODE trained on just 10% of the full simulation. While this last finding highlights an area for future work, our results suggest that using NODEs to enrich limited data and enable symbolic regression to infer physical laws represents a promising new approach for scientific discovery.

Figures (3)

• Figure 1: The red rectangle in both plots represents the training region. (a) The MSE heatmap for Model B (cart-pole). The low-error zones outside the training region highlight the model's ability to extrapolate. This occurs because the model learned the underlying dynamics from trajectories within the training set, allowing it to accurately predict other points along the same dynamic path. (b) The cart-pole phase space. The colour represents the magnitude of the angular speed and acceleration.
• Figure 2: (a) Cart-pole. Model A Time Extrapolation Performance. Model A, trained on data from only the first second, accurately predicts the system's dynamics for an interpolated initial condition, [1.4, 5]. The model demonstrates strong generalization by successfully extrapolating five times the length of the training data. (b) Bio-model. Results from Model 2A, which was trained solely on two up-shift datasets. The model successfully predicts a down-shift ($\nu_i$ = 5.95 to $\nu_f$ = 3.78), an event it was never exposed to in training. While small amplitude errors (less than 5%) are observed, the model effectively learns and extrapolates the underlying shift dynamics.
• Figure 3: Impact of sampling frequency on model performance. The legend shows the time regions for MSE calculation. The 8-hour MSE is consistent across all models, demonstrating that long-term predictions are achievable with sparse data. In contrast, the 1-hour MSE rises sharply for the two lowest sampling frequencies, as the model’s fit is highly sensitive to noise when trained on only a handful of points. Errobars indicate the standard deviation over 10 independent runs.