Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Variationally Correct Neural Residual Regression for Parametric PDEs: On the Viability of Controlled Accuracy

Markus Bachmayr, Wolfgang Dahmen, Mathias Oster

Abstract

This paper is about learning the parameter-to-solution map for systems of partial differential equations (PDEs) that depend on a potentially large number of parameters covering all PDE types for which a stable variational formulation (SVF) can be found. A central constituent is the notion of variationally correct residual loss function meaning that its value is always uniformly proportional to the squared solution error in the norm determined by the SVF, hence facilitating rigorous a posteriori accuracy control. It is based on a single variational problem, associated with the family of parameter dependent fiber problems, employing the notion of direct integrals of Hilbert spaces. Since in its original form the loss function is given as a dual test norm of the residual a central objective is to develop equivalent computable expressions. A first critical role is played by hybrid hypothesis classes, whose elements are piecewise polynomial in (low-dimensional) spatio-temporal variables with parameter-dependent coefficients that can be represented, e.g. by neural networks. Second, working with first order SVFs, we distinguish two scenarios: (i) the test space can be chosen as an $L_2$-space (e.g. for elliptic or parabolic problems) so that residuals live in $L_2$ and can be evaluated directly; (ii) when trial and test spaces for the fiber problems (e.g. for transport equations) depend on the parameters, we use ultraweak formulations. In combination with Discontinuous Petrov Galerkin concepts the hybrid format is then instrumental to arrive at variationally correct computable residual loss functions. Our findings are illustrated by numerical experiments representing (i) and (ii), namely elliptic boundary value problems with piecewise constant diffusion coefficients and pure transport equations with parameter dependent convection field.

Variationally Correct Neural Residual Regression for Parametric PDEs: On the Viability of Controlled Accuracy

Abstract

This paper is about learning the parameter-to-solution map for systems of partial differential equations (PDEs) that depend on a potentially large number of parameters covering all PDE types for which a stable variational formulation (SVF) can be found. A central constituent is the notion of variationally correct residual loss function meaning that its value is always uniformly proportional to the squared solution error in the norm determined by the SVF, hence facilitating rigorous a posteriori accuracy control. It is based on a single variational problem, associated with the family of parameter dependent fiber problems, employing the notion of direct integrals of Hilbert spaces. Since in its original form the loss function is given as a dual test norm of the residual a central objective is to develop equivalent computable expressions. A first critical role is played by hybrid hypothesis classes, whose elements are piecewise polynomial in (low-dimensional) spatio-temporal variables with parameter-dependent coefficients that can be represented, e.g. by neural networks. Second, working with first order SVFs, we distinguish two scenarios: (i) the test space can be chosen as an -space (e.g. for elliptic or parabolic problems) so that residuals live in and can be evaluated directly; (ii) when trial and test spaces for the fiber problems (e.g. for transport equations) depend on the parameters, we use ultraweak formulations. In combination with Discontinuous Petrov Galerkin concepts the hybrid format is then instrumental to arrive at variationally correct computable residual loss functions. Our findings are illustrated by numerical experiments representing (i) and (ii), namely elliptic boundary value problems with piecewise constant diffusion coefficients and pure transport equations with parameter dependent convection field.
Paper Structure (35 sections, 11 theorems, 160 equations, 5 figures, 4 tables)

This paper contains 35 sections, 11 theorems, 160 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Beyond projection-based model reduction
  3. Model reduction by empirical risk minimization
  4. Limitations of basic PINN formulations
  5. Proportionality of error and residual
  6. Relation to existing results
  7. Hybrid approximation format
  8. Novelty and organization of the material
  9. Variational Formulations and Residual Loss Functions
  10. Parametric fiber problems
  11. Direct integrals
  12. The lifted problem and its well-posedness
  13. Obstructions and main goal
  14. Computable Variationally Correct Loss Functions
  15. Conceptual prerequisites
  16. ...and 20 more sections

Key Result

Theorem 2.3

Let $(\mathbb{W}_p)_{p \in P}$ be a family of separable Hilbert spaces such that $\prod_{p\in P}\mathbb{W}_p$ contains a fundamental sequence of $\mu$-measurable sections. Then the spaces $L_2\bigl(P, (\mathbb{W}_p)_{p \in P} \bigr)$ defined by where elements that agree $\mu$-almost everywhere are identified, endowed with the inner product are Hilbert spaces.

Figures (5)

  • Figure 1: Snapshot of the LSG solution, the prediction by the neural network (with rank 60 and 15 layers trained on 1000 samples) and the pointwise absolute value of the difference for a level four refinement evaluated for the parameter $p = [0.65,1.45,1.45,0.65]$ corresponding to a checkerboard configuration.
  • Figure 2: Snapshot of the LSG solution, the prediction by the neural network (with rank 60 and 15 layers trained on 1000 samples) and the pointwise absolute value of the difference for a level four refinement and given parameter $p = [0.53,1.09,0.84,0.82]$
  • Figure 3: Visualization of the relative dual norm of the residual of the prediction by the neural network versus the DPG solution for a network with rank 20 trained on 1000 samples.
  • Figure 4: Snapshot for $p = 0.2$ of piecewise constant elements for the DPG solution and the prediction by the neural network with rank 20 and 15 layers, trained on 1000 samples for level 6.
  • Figure 5: Snapshot for $p = 0.1$ of piecewise constant elements for the DPG solution and the prediction by the neural network with rank 20 and 15 layers, trained on 1000 samples for level 6.

Theorems & Definitions (35)

  • Remark 2.1
  • Definition 2.2
  • Theorem 2.3
  • Remark 2.4
  • Theorem 2.5
  • proof
  • Proposition 2.7
  • Remark 2.9
  • Theorem 2.10
  • proof
  • ...and 25 more