Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Error estimation for physics-informed neural networks with implicit Runge-Kutta methods

Jochen Stiasny, Spyros Chatzivasileiadis

TL;DR

This work proposes to use the NN's predictions in a high-order implicit Runge-Kutta (IRK) method and finds that this error estimate highly correlates with the NN's prediction error and that increasing the order of the IRK method improves this estimate.

Abstract

The ability to accurately approximate trajectories of dynamical systems enables their analysis, prediction, and control. Neural network (NN)-based approximations have attracted significant interest due to fast evaluation with good accuracy over long integration time steps. In contrast to established numerical approximation schemes such as Runge-Kutta methods, the estimation of the error of the NN-based approximations proves to be difficult. In this work, we propose to use the NN's predictions in a high-order implicit Runge-Kutta (IRK) method. The residuals in the implicit system of equations can be related to the NN's prediction error, hence, we can provide an error estimate at several points along a trajectory. We find that this error estimate highly correlates with the NN's prediction error and that increasing the order of the IRK method improves this estimate. We demonstrate this estimation methodology for Physics-Informed Neural Network (PINNs) on the logistic equation as an illustrative example and then apply it to a four-state electric generator model that is regularly used in power system modelling.

Error estimation for physics-informed neural networks with implicit Runge-Kutta methods

TL;DR

This work proposes to use the NN's predictions in a high-order implicit Runge-Kutta (IRK) method and finds that this error estimate highly correlates with the NN's prediction error and that increasing the order of the IRK method improves this estimate.

Abstract

The ability to accurately approximate trajectories of dynamical systems enables their analysis, prediction, and control. Neural network (NN)-based approximations have attracted significant interest due to fast evaluation with good accuracy over long integration time steps. In contrast to established numerical approximation schemes such as Runge-Kutta methods, the estimation of the error of the NN-based approximations proves to be difficult. In this work, we propose to use the NN's predictions in a high-order implicit Runge-Kutta (IRK) method. The residuals in the implicit system of equations can be related to the NN's prediction error, hence, we can provide an error estimate at several points along a trajectory. We find that this error estimate highly correlates with the NN's prediction error and that increasing the order of the IRK method improves this estimate. We demonstrate this estimation methodology for Physics-Informed Neural Network (PINNs) on the logistic equation as an illustrative example and then apply it to a four-state electric generator model that is regularly used in power system modelling.
Paper Structure (17 sections, 23 equations, 5 figures)

This paper contains 17 sections, 23 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Neural network-based integration:
  3. Approaches for analysing the error of PINNs:
  4. Proposed approach:
  5. Paper structure:
  6. Approximating the trajectory of a dynamical system
  7. Implicit Runge-Kutta (IRK) methods
  8. Neural-Network-based approximation
  9. Error estimates of PINN predictions with IRK methods
  10. Estimation of $\bm{\delta}^{(i)}$:
  11. Logistic equation - illustration with an analytical solution
  12. Approximating a trajectory of the logistic equation
  13. Visualising the stages of an IRK scheme
  14. Estimating the stage error
  15. Error estimation for a power system generator model
  16. ...and 2 more sections

Figures (5)

  • Figure 1: Predictions from different numerical integration schemes for a time step of size $h$.
  • Figure 2: Analysis of the prediction accuracy of IRK stages for a time step of size $h$.
  • Figure 3: Correlation between the estimated stage error $\delta^{(i)}$ and the corresponding prediction error $|\hat{x}(t_0 + c_i h) - x(t_0 + c_i h)|$ for IRK schemes with 3, 8, and 32 stages. The black dots represent a time step size of $h=5$ and the red dots a time step size of $h=10$.
  • Figure 4: Consistency of stage prediction for different orders of the IRK schemes. The points indicate the error estimates $\delta^{(i)}$ for a range of values of $h$. The exact error $|\hat{\bm{x}} - \bm{x}|$ is shown in orange. For low order IRK schemes, the error estimates for a given time $c_i h$ clearly contradict each other. For a high-order IRK scheme all error estimates form a consistent estimate that is in line with the exact error characteristics.
  • Figure 5: Error distribution of the PINN predictions with respect to the time step size $h$ for a two-axis generator model. We compare the estimated errors $\left\lVert\bm{\delta}^{(i)}\right\rVert_{2}$ to the prediction error evaluated with a highly accurate solver. As the error depends also on the initial condition, the grey shadings correspond to the errors for 100%, 80%, and 50% of these trajectories.