Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural network-enhanced integrators for simulating ordinary differential equations

Amine Othmane, Kathrin Flaßkamp

TL;DR

This work tackles the challenge of efficiently solving parameterized ODEs with high accuracy by augmenting traditional time-stepping schemes with neural network learned corrections to the local truncation error. A general framework trains an NN to approximate the leading local error term of explicit Runge–Kutta methods and injects a corrective term into the discrete flow, with backward error analysis ensuring the enhanced method remains faithful to the underlying dynamics. Embedded Runge–Kutta strategies are employed to mitigate generalization risk, providing a safety net that reverts to classical solvers when the NN's prediction is unreliable. The methodology is validated on a wind turbine dynamics model drawn from OpenFast CADynTurb, showing improved speed-accuracy trade-offs and robustness, highlighting the practical potential of combining classical numerical analysis with data-driven error correction for complex, high-dimensional systems.

Abstract

Numerous applications necessitate the computation of numerical solutions to differential equations across a wide range of initial conditions and system parameters, which feeds the demand for efficient yet accurate numerical integration methods.This study proposes a neural network (NN) enhancement of classical numerical integrators. NNs are trained to learn integration errors, which are then used as additive correction terms in numerical schemes. The performance of these enhanced integrators is compared with well-established methods through numerical studies, with a particular emphasis on computational efficiency. Analytical properties are examined in terms of local errors and backward error analysis. Embedded Runge-Kutta schemes are then employed to develop enhanced integrators that mitigate generalization risk, ensuring that the neural network's evaluation in previously unseen regions of the state space does not destabilize the integrator. It is guaranteed that the enhanced integrators perform at least as well as the desired classical Runge-Kutta schemes. The effectiveness of the proposed approaches is demonstrated through extensive numerical studies using a realistic model of a wind turbine, with parameters derived from the established simulation framework OpenFast.

Neural network-enhanced integrators for simulating ordinary differential equations

TL;DR

This work tackles the challenge of efficiently solving parameterized ODEs with high accuracy by augmenting traditional time-stepping schemes with neural network learned corrections to the local truncation error. A general framework trains an NN to approximate the leading local error term of explicit Runge–Kutta methods and injects a corrective term into the discrete flow, with backward error analysis ensuring the enhanced method remains faithful to the underlying dynamics. Embedded Runge–Kutta strategies are employed to mitigate generalization risk, providing a safety net that reverts to classical solvers when the NN's prediction is unreliable. The methodology is validated on a wind turbine dynamics model drawn from OpenFast CADynTurb, showing improved speed-accuracy trade-offs and robustness, highlighting the practical potential of combining classical numerical analysis with data-driven error correction for complex, high-dimensional systems.

Abstract

Numerous applications necessitate the computation of numerical solutions to differential equations across a wide range of initial conditions and system parameters, which feeds the demand for efficient yet accurate numerical integration methods.This study proposes a neural network (NN) enhancement of classical numerical integrators. NNs are trained to learn integration errors, which are then used as additive correction terms in numerical schemes. The performance of these enhanced integrators is compared with well-established methods through numerical studies, with a particular emphasis on computational efficiency. Analytical properties are examined in terms of local errors and backward error analysis. Embedded Runge-Kutta schemes are then employed to develop enhanced integrators that mitigate generalization risk, ensuring that the neural network's evaluation in previously unseen regions of the state space does not destabilize the integrator. It is guaranteed that the enhanced integrators perform at least as well as the desired classical Runge-Kutta schemes. The effectiveness of the proposed approaches is demonstrated through extensive numerical studies using a realistic model of a wind turbine, with parameters derived from the established simulation framework OpenFast.

Paper Structure

This paper contains 25 sections, 2 theorems, 41 equations, 6 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. State of the art
  3. Contribution
  4. Application case study
  5. Outline
  6. Problem statement and background
  7. Neural network enhanced integrators
  8. Definition of neural networks
  9. Learning problem for local error correction
  10. Enhanced solvers
  11. Analysis of enhanced integrators
  12. Enhanced integrators with mitigated generalization risk
  13. Local error approximation:
  14. Mitigating the generalization risk of neural networks:
  15. Efficient implementation of hybrid enhanced solvers
  16. ...and 10 more sections

Key Result

Proposition 1

Consider the initial value problem eq:ODE and an artificial neural network $\mathcal{N}_\theta : \mathbb{R}^{n_x} \times \mathbb{R}^n_p \;\to\;\mathbb{R}^{n_x}.$ Let $\boldsymbol{\eta}_{h}$ be a discrete flow of a numerical scheme of order $p$ with step size $h$, and denote by $L_{\boldsymbol{\eta}_ with the local error $r$ defined in eq:local_error. Consider the error with $\tilde{\boldsymbol{x}

Figures (6)

  • Figure 1: Time evolution of the state component $\boldsymbol{x}_{\mathrm{FA}}$ for three different wind speeds $v_{\text{wind}}$.
  • Figure 2: Comparison of four solvers with respect to computational effort $\delta_t$ from \ref{['eq:timing']} and normalized global error $\delta_e$ from \ref{['eq:global_error']}. Each point shows the median $\delta_t$ over $10^6$ evaluations and $\delta_e$ over 200 simulations with different speeds in the interval $[5,10]m\per s$.
  • Figure 3: Normalized global error $\delta_e$, as defined in \ref{['eq:global_error']}, computed over 200 simulations with different speeds in the interval $[5,10]m\per s$. The time step for the neural network based solver is $h = 10ms$. The corresponding computational cost $\delta_t$, defined in \ref{['eq:timing']}, is evaluated over $10^6$ function calls. The isolated points correspond to outliers. The two RK3 schemes have step sizes of $15ms$ and $10ms$.
  • Figure 4: Normalized global error $\delta_e$, as defined in \ref{['eq:global_error']}, computed over 200 simulations with different speeds in the interval $[5,10]m\per s$. The time step for the neural network based solver is $h = 5ms$. The corresponding computational cost $\delta_t$, defined in \ref{['eq:timing']}, is evaluated over $10^6$ function calls. The two RK3 schemes have step sizes of $15ms$ and $10ms$
  • Figure 5: Normalized global error $\delta_e$, as defined in \ref{['eq:global_error']}, computed over 200 simulations with varying wind speeds in the interval $[5,10]m\per s$. The values are such that $75\%$ of them are in the interval $[5,20]m\per s$. The time step for the neural network based solver is $h = 10ms$. The corresponding computational cost $\delta_t$, defined in \ref{['eq:timing']}, is evaluated over $10^6$ function calls. The isolated points correspond to outliers. The two RK3 schemes have step sizes of $15ms$ and $10ms$
  • ...and 1 more figures

Theorems & Definitions (8)

  • Example 1
  • Example 2
  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Remark 1
  • Example 3