Data-driven Methods for Delay Differential Equations

TL;DR

The chapter surveys data-driven strategies for time-delay dynamics, contrasting SINDy-based methods (E-SINDy and P-SINDy) with neural delay differential equations (NDDEs). It explains how to extend SINDy to discrete-delayed systems, either directly (E-SINDy) or via ODE discretization with pseudospectral collocation (P-SINDy), and discusses external optimization vs internal sparse regression for learning delays. NDDEs are presented as gradient-based, trainable-delay models that can learn delays and dynamics simultaneously, with derivative and simulation loss options and practical training algorithms. Through tests on classical DDEs (delay logistic, Mackey-Glass) and delayed Rössler/neurons/climate models, the chapter highlights tradeoffs between interpretability, computational cost, and predictive accuracy, offering guidance on when to prefer each approach and pointing to future directions in data-driven time-delay modeling.

Abstract

Data-driven methodologies are nowadays ubiquitous. Their rapid development and spread have led to applications even beyond the traditional fields of science. As far as dynamical systems and differential equations are concerned, neural networks and sparse identification tools have emerged as powerful approaches to recover the governing equations from available temporal data series. In this chapter we first illustrate possible extensions of the sparse identification of nonlinear dynamics (SINDy) algorithm, originally developed for ordinary differential equations (ODEs), to delay differential equations (DDEs) with discrete, possibly multiple and unknown delays. Two methods are presented for SINDy, one directly tackles the underlying DDE and the other acts on the system of ODEs approximating the DDE through pseudospectral collocation. We also introduce another way of capturing the dynamics of DDEs using neural networks and trainable delays in continuous time, and present the training algorithms developed for these neural delay differential equations (NDDEs). The relevant MATLAB implementations for both the SINDy approach and for the NDDE approach are provided. These approaches are tested on several examples, including classical systems such as the delay logistic and the Mackey-Glass equation, and directly compared to each other on the delayed Rössler system. We provide insights on the connection between the approaches and future directions on developing data-driven methods for time delay systems.

Figures (13)

• Figure 1: True (gray) and E-SINDy trajectory (cyan) of the delay logistic equation \ref{['eq:logistic']} for ${r=1.8}$, ${K=10}$ and ${\tau=1}$, reconstructed with ${m=10}$ uniform (left) and random (right) training samples (red dots) in ${[0,18]}$, and tested in ${[18,30]}$.
• Figure 2: (left) External optimization of the unknown delay $\tau$ and the Hill exponent $\alpha$ for the Mackey-Glass equation \ref{['eq:mg']} using BF. Color indicates the ${\textrm{RMSE}_{x'}}$ over a ${100 \times 100}$ grid in the ${(\tau,\alpha)}$ plane, the nominal values are ${(\tau,\alpha)=(1, 9.6)}$, and the optimal point ${(\widehat{\tau},\widehat{\alpha})=(1.0020, 9.5475)}$ is marked by red circle. (right) True (gray) and E-SINDy (red) trajectories after external optimization.
• Figure 3: (left) Reconstructed trajectory for the two-neuron model \ref{['eq:neuron']} showing the true trajectory (gray), E-SINDy trajectory obtained with BO (red), P-SINDy trajectory of collocation degree ${M=10}$ obtained with PS (blue). The true delay values are ${(\tau_\mathrm{s},\tau_1,\tau_2)=(1.5, 2, 2)}$, the optimized delays values for E-SINDy are ${(\widehat{\tau}_s,\widehat{\tau}_1,\widehat{\tau}_2)=(1.5136, 1.9478,2.0598)}$ while for the P-SINDy approach the recovered delay is ${\bar{\tau}= 2.0019}$. (right) Delay recovery of delays for E-SINDy ${(\widehat{\tau}_{s},\widehat{\tau}_{1},\widehat{\tau}_{2})}$ using PS.
• Figure 4: (left) $\textrm{RMSE}_{x^\prime}$ for Mackey-Glass \ref{['eq:mg']} for E-SINDy and P-SINDy with ${M=10}$ as a function of the total number of SINDy calls using Bayesian optimization (BO) and Paricle Swarm (PS). (right) Trajectory reconstruction for \ref{['eq:mg']} showing the true trajectory (gray), E-SINDy trajectory using BO (red) and P-SINDy trajectory using ${M=10}$ and PS (blue). For the true model, the nominal values are ${(\tau,\alpha)=(1, 9.6)}$. E-SINDy with BO identify ${(\widehat{\tau},\widehat{\alpha})=(0.9996, 9.4969)}$, whereas P-SINDy with ${M=10}$ and PS identify ${(\widehat{\tau},\widehat{\alpha})=(1.0000, 9.6001)}$.
• Figure 5: Scatter plots of optimized unknown delays $(\widehat{\tau}_s,\,\widehat{\tau}_1,\,\widehat{\tau}_2)$ using particle swarm (PS) for the two-neuron model \ref{['eq:neuron']}, color-coded by RMSE$_{x^\prime}$ values. Red stars mark the true values of the parameters.

Theorems & Definitions (5)

• Example 2.1
• Example 2.2
• Example 2.3
• Example 3.1
• Example 4.1