Machine learning identifies nullclines in oscillatory dynamical systems

TL;DR

Cline is introduced, a neural network-based method that uncovers the hidden structure of nullclines from oscillatory time series data and overcomes challenges such as multiple time scales and strong nonlinearities while producing interpretable results convertible into symbolic differential equations.

Abstract

We introduce CLINE (Computational Learning and Identification of Nullclines), a neural network-based method that uncovers the hidden structure of nullclines from oscillatory time series data. Unlike traditional approaches aiming at direct prediction of system dynamics, CLINE identifies static geometric features of the phase space that encode the (non)linear relationships between state variables. It overcomes challenges such as multiple time scales and strong nonlinearities while producing interpretable results convertible into symbolic differential equations. We validate CLINE on various oscillatory systems, showcasing its effectiveness.

Figures (4)

• Figure 1: CLINE uses neural networks to extract hidden nullcline structures from time series data. Feed-forward neural networks are trained on known time series data and the derivative of a state variable of interest (e.g., $u, u_t$ or $v, v_t$) to predict the other variable ($v$ or $u$). Setting the derivative input to zero allows the network to learn the underlying nullcline structure, $f(u,v)$ or $g(u,v)$. This information can then be transformed into symbolic equations using methods such as SINDyBrunton2016.
• Figure 2: Nullcline structures are accurately reconstructed when using appropriate input variables. (a) A representative time series of the FHN model alongside its phase portrait. The model consists of a linear nullcline ($v_t = g(u,v)$) and an S-shaped cubic nullcline ($u_t = f(u,v)$). (b) After normalizing the state variables, the training process enables the model to converge to the ground-truth (GT) limit cycle (LC), as demonstrated by the reconstructed S-shaped nullcline (NC). (c) Prediction success depends on the choice of input variables for training the deep learning model. If unsuitable inputs are used (e.g., $v$ and $u_t$ to infer $f(u,v)$), CLINE fails to correctly identify the GT limit cycle and nullcline due to poorly separated input variable limit cycles, as shown in the insets. Conversely, with appropriate input variables, nullcline identification is successful.
• Figure 3: CLINE robustly identifies nullclines despite time scale separation and model complexity. (a) In the FHN model, CLINE accurately reconstructs nullclines ($f(u,v) = 0$) across different levels of time scale separation. Stronger separation (low $\varepsilon$) improves reconstruction as the limit cycle explores more of the phase space. (b) CLINE also generalizes to more complex models, such as the Bicubic model with two S-shaped nullclines (Eq. \ref{['eq:bicubic']}) and a gene expression model with S-shaped and ultrasensitive nullclines.
• Figure 4: CLINE accurately identifies nullclines in DDEs when the delay is close to the true value. (a) In the DDE from Eq. (\ref{['eq:DDE']}), the second variable is introduced as $v = u(t - \tau)$. This creates a nonlinear and rational nullcline well approximated by CLINE. (b) When the chosen delay $\tau$ is close to the ground-truth delay $\tau_{\text{GT}} = 10$ (green dot), specifically within $\tau_{\text{GT}} - 1 \leq \tau \leq \tau_{\text{GT}} + 2$, the phase space and identified nullcline remain accurate with low mean squared error (MSE) and small variations. However, for larger deviations (e.g., $\tau = \tau_{\text{GT}} - 5$ or $\tau = \tau_{\text{GT}} + 10$), CLINE fails to recover the correct nullcline, leading to increased MSE (error bars over five iterations). Errors are particularly pronounced when $\tau$ corresponds to $T/4$ or $3T/4$ of the system's period $T$, as seen for $\tau = \tau_{\text{GT}} - 5$, where the limit cycle lacks distinct maxima and minima.