Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Trustworthy AI in numerics: On verification algorithms for neural network-based PDE solvers

Emil Haugen, Alexei Stepanenko, Anders C. Hansen

TL;DR

This work tackles the challenge of trustworthy NN-based PDE solvers by proposing a posteriori verification that delivers $\varepsilon$-accuracy guarantees through residual-based bounds and computable $L^p$ norms of neural network derivatives. The authors develop activation- and derivative-bound machinery to enable rigorous quadrature bounds, derive first-, second-, and third-order derivative bounds, and construct two verification algorithms (local and global) that certify NN solutions independently of the training process. The framework is demonstrated for the heat equation, combining energy estimates with robust quadrature of residuals $\phi_{\mathcal{P}}$ and $\phi_{0}$, and is supported by numerical experiments showing practical feasibility. By providing computable a posteriori certificates, the approach advances trustworthy AI in PDE contexts with potential extensions to broader PDEs and operator learning paradigms.

Abstract

We present new algorithms for a posteriori verification of neural networks (NNs) approximating solutions to PDEs. These verification algorithms compute accurate estimates of $L^p$ norms of NNs and their derivatives. When combined with residual bounds for specific PDEs, the algorithms provide guarantees of $\eps$-accuracy (in a suitable norm) with respect to the true, but unknown, solution of the PDE -- for arbitrary $\eps >0$. In particular, if the NN fails to meet the desired accuracy, our algorithms will detect that and reject it, whereas any NN that passes the verification algorithms is certified to be $\eps$-accurate. This framework enables trustworthy algorithms for NN-based PDE solvers, regardless of how the NN is initially computed. Such a posteriori verification is essential, since a priori error bounds in general cannot guarantee the accuracy of computed solutions, due to algorithmic undecidability of the optimization problems used to train NNs.

Trustworthy AI in numerics: On verification algorithms for neural network-based PDE solvers

TL;DR

This work tackles the challenge of trustworthy NN-based PDE solvers by proposing a posteriori verification that delivers -accuracy guarantees through residual-based bounds and computable norms of neural network derivatives. The authors develop activation- and derivative-bound machinery to enable rigorous quadrature bounds, derive first-, second-, and third-order derivative bounds, and construct two verification algorithms (local and global) that certify NN solutions independently of the training process. The framework is demonstrated for the heat equation, combining energy estimates with robust quadrature of residuals and , and is supported by numerical experiments showing practical feasibility. By providing computable a posteriori certificates, the approach advances trustworthy AI in PDE contexts with potential extensions to broader PDEs and operator learning paradigms.

Abstract

We present new algorithms for a posteriori verification of neural networks (NNs) approximating solutions to PDEs. These verification algorithms compute accurate estimates of norms of NNs and their derivatives. When combined with residual bounds for specific PDEs, the algorithms provide guarantees of -accuracy (in a suitable norm) with respect to the true, but unknown, solution of the PDE -- for arbitrary . In particular, if the NN fails to meet the desired accuracy, our algorithms will detect that and reject it, whereas any NN that passes the verification algorithms is certified to be -accurate. This framework enables trustworthy algorithms for NN-based PDE solvers, regardless of how the NN is initially computed. Such a posteriori verification is essential, since a priori error bounds in general cannot guarantee the accuracy of computed solutions, due to algorithmic undecidability of the optimization problems used to train NNs.

Paper Structure

This paper contains 33 sections, 12 theorems, 76 equations, 3 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Necessity of a posteriori error bounds -- Undecidability in optimization
  3. Novelty of contributions and future work
  4. Existing literature
  5. Neural networks and rigorous verification
  6. Approximation error through residual bounds in PDEs
  7. The general numerical problem and choosing the quadrature
  8. Algorithms for derivative estimates of neural networks
  9. First order derivative bounds
  10. Computing activation bounds
  11. From activation bounds to first derivative bounds
  12. Higher derivatives of neural networks
  13. Activation bounds for higher derivatives
  14. Second derivatives
  15. Third derivatives
  16. ...and 18 more sections

Key Result

Proposition 2.1

Let $f\colon \mathbb{R}^d \to \mathbb{R}$ and $\alpha$ be a multi-index such that $\partial^\alpha f \in C^1(\mathbb{R}^d)$. Let $y \in \mathbb{R}^d, \varepsilon > 0$ and define $\mathcal{B} \coloneqq \mathcal{B}_\varepsilon(y)$ as in eq:Omega-union. Then the following estimate holds.

Figures (3)

  • Figure 1: ( Rigorously estimating the $L^2$ norm of an NN.) Outputs from our algorithms estimating the $L^2$ norm of an NN $f \in {\mathcal{F}}_2^{(3)}$ with two hidden layers of width $100$, trained to solve the heat equation \ref{['eq:heat']} in the scenario $T=1, g(x) = \sin(\pi x), \kappa = 1/\pi^2$ using $\varepsilon=0.01$. Left: local quadrature estimates $\varepsilon^2 f(y)^2$ as in \ref{['eq:midpoint']}. Right: local error bounds $R_2(y, \varepsilon)$ as in \ref{['eq:L2-box-res']}. Summing over all grid points yields global estimates $I_p(f)$ and bounds $\mathcal{R}_{\mathcal{P}}$ as shown in \ref{['table:Lp-norm-example']}.
  • Figure 2: ( Error bounds for the heat equation, PDE residual). Rigorous quadrature computations for the $L^2$ norm of the PDE residual $\phi_{\mathcal{P}}$. Local quadrature estimates $\varepsilon^d \phi_{\mathcal{P}}(y)^2$ as in \ref{['eq:heat-spacetime-norm']} and corresponding local error bounds $R_{\mathcal{P}}(y, \varepsilon)$ as in \ref{['eq:heat-quad-error-bound']} computed with $\varepsilon=0.01$. Summing over all $y \in Y_{\mathrm{grid}}$ yields the global estimate $I_2(\phi_{\mathcal{P}})$ and error bound $\mathcal{R}_{\mathcal{P}}$ which are shown in \ref{['table:pde']}.
  • Figure 3: ( Error bounds for the heat equation, initial data residual). Rigorous quadrature computations for the $L^2$ norm of the initial data residual $\phi_{0}$. Local quadrature estimates $\varepsilon^{d-1}\phi_{0}(y)^2$ as in \ref{['eq:heat-initial-norm']} and corresponding local error bounds $R_0(y, \varepsilon)$ as in \ref{['eq:heat-initial-norm-bound']} computed using our algorithms with $\varepsilon=0.01$. Summing over all $y \in Y_0$ yields the global estimate $I_2(\phi_{0})$ and error bound $\mathcal{R}_0$ which are shown in \ref{['table:initial']}.

Theorems & Definitions (27)

  • Remark 1.1: Non-computability even in the randomized case
  • Remark 1.2
  • Remark 2.1: Other activation functions
  • Proposition 2.1: Local quadrature error
  • proof
  • Remark 2.2: On choice of quadrature
  • Remark 2.3: Operator learning
  • Example 2.1: $L^p$ norm
  • Remark 3.1
  • Lemma 3.1
  • ...and 17 more