Certified machine learning: A posteriori error estimation for physics-informed neural networks

TL;DR

This paper derives an a posteriori error estimator for physics-informed neural networks (PINNs) solving ODEs, showing the true prediction error ||e(t)|| can be bounded by the residual and initial-condition mismatch, e.g., ||e(t)|| ≤ e^{Lt} ||x0 - x_hat(0)|| + I(t,δ). The estimator is computable via trapezoidal integration and a residual-smoothing trick, and can be approximated by a data-driven Error NN to reduce cost. Two tests on a 1D linear ODE and a nonlinear inverted pendulum demonstrate that the bound is a rigorous upper bound and that initialization error dominates early on. The framework enables certified PINN predictions and suggests extensions to PDEs and further methodological refinements such as activation choices and η-weighting.

Abstract

Physics-informed neural networks (PINNs) are one popular approach to incorporate a priori knowledge about physical systems into the learning framework. PINNs are known to be robust for smaller training sets, derive better generalization problems, and are faster to train. In this paper, we show that using PINNs in comparison with purely data-driven neural networks is not only favorable for training performance but allows us to extract significant information on the quality of the approximated solution. Assuming that the underlying differential equation for the PINN training is an ordinary differential equation, we derive a rigorous upper limit on the PINN prediction error. This bound is applicable even for input data not included in the training phase and without any prior knowledge about the true solution. Therefore, our a posteriori error estimation is an essential step to certify the PINN. We apply our error estimator exemplarily to two academic toy problems, whereof one falls in the category of model-predictive control and thereby shows the practical use of the derived results.

Figures (8)

• Figure 1: Training sequence for the error neural network based on conventional evaluations of the contributions of Lemma \ref{['lemma::trpz_rule']} (grey node in the top right corner). The two neural networks, the one reflecting the evolution of the system over time and the other one describing the error are called PINN and Error NN, respectively.
• Figure 2: Dependency of computed $K$ on selection of $\mu$ for a neural network with tanh activation function.
• Figure 3: Actual absolute error of the learned solution (red line) and the predicted error by a posteriori error estimation (orange line). The first data point of $E_\mathrm{PI}$ (green dashed line) is out of scope of the figure because it is evaluated to zero at time zero. This is reasonable, since the fulfillment of the ODE plays no role if no time has passed.
• Figure 4: Predicted error with the a posteriori error estimator evaluated by numerical integration (orange solid line) in contrast to error prediction of the neural network (blue lines) trained with artificially generated data (data set size 100). Here, a neural network with 2 layers and 4 neurons in each layer is used. The difference between the two depicted curves is, whether underestimating the error was penalized more than overestimating it. In case of $E_\mathrm{NN, weighted}$ (blue dashed line) it was penalized by multiplying the loss with 1000 for underestimation, while this was 1 for $E_\mathrm{NN, notweighted}$ (blue dotted line).
• Figure 5: Inverted pendulum as described by \ref{['eq::ODE_inv_pendulum']} with the variables of interest being the angle of the pendulum $\phi$ and displacement of the cart $s$. The parameters $m= 0.3553\, \mathrm{kg}$, $a = 0.42\,\mathrm{m}$, and $J= 0.0361\, \mathrm{kg}/\mathrm{m}^2$ denote the (point) mass of the cart, the distance between the center of mass of the cart and the pendulum $a$, and the mass moment of inertia, respectively.

Theorems & Definitions (12)

• Theorem 3.1
• proof
• Remark 3.2
• Theorem 3.3
• proof
• Remark 3.4
• Lemma 3.5
• proof
• Remark 4.1
• Remark 5.1