Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Kernel Methods in the Deep Ritz framework: Theory and practice

Hendrik Kleikamp, Tizian Wenzel

Abstract

In this contribution, kernel approximations are applied as ansatz functions within the Deep Ritz method. This allows to approximate weak solutions of elliptic partial differential equations with weak enforcement of boundary conditions using Nitsche's method. A priori error estimates are proven in different norms leveraging both standard results for weak solutions of elliptic equations and well-established convergence results for kernel methods. This availability of a priori error estimates renders the method useful for practical purposes. The procedure is described in detail, meanwhile providing practical hints and implementation details. By means of numerical examples, the performance of the proposed approach is evaluated numerically and the results agree with the theoretical findings.

Kernel Methods in the Deep Ritz framework: Theory and practice

Abstract

In this contribution, kernel approximations are applied as ansatz functions within the Deep Ritz method. This allows to approximate weak solutions of elliptic partial differential equations with weak enforcement of boundary conditions using Nitsche's method. A priori error estimates are proven in different norms leveraging both standard results for weak solutions of elliptic equations and well-established convergence results for kernel methods. This availability of a priori error estimates renders the method useful for practical purposes. The procedure is described in detail, meanwhile providing practical hints and implementation details. By means of numerical examples, the performance of the proposed approach is evaluated numerically and the results agree with the theoretical findings.
Paper Structure (16 sections, 7 theorems, 37 equations, 7 figures, 1 table)

This paper contains 16 sections, 7 theorems, 37 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Theoretical background
  3. Variational formulation for elliptic PDEs and energy minimization
  4. Kernel methods and approximation error estimates
  5. Application of kernel approximations in the deep Ritz method
  6. Description of the method
  7. A priori error estimation
  8. Practical aspects
  9. Numerical experiments
  10. Implementational details
  11. Diffusion equation with a solution of high regularity
  12. Singular solution on a non-convex domain
  13. Conclusion and outlook
  14. Funding
  15. Competing interests
  16. ...and 1 more sections

Key Result

Theorem 1

Let $V_h\subset V$ be a closed subspace of $V$. Further, let $a\colon V\times V\to\mathbb{R}$ be a symmetric, continuous and coercive bilinear form and $l\in V'$ a continuous linear functional. We denote by $u\in V$ the weak solution in $V$ such that it holds $a(u,v)=l(v)$ for all $v\in V$, and by $ holds, where $\gamma>0$ is the continuity constant, see equ:definition-continuity, and $\alpha>0$ d

Figures (7)

  • Figure 1: Relative errors in the $L^2$-norm (left) and the $H^1$-norm (right) with respect to the mesh norm $h$ for Matérn kernels of smoothness $\nu \in \{1/2,3/2,5/2\}$ in the example with a smooth solution. The dashed lines indicate the error decays of the interpolation of the exact solution by the respective kernel. The estimated convergence rates for the interpolated solutions are shown as black lines.
  • Figure 2: Relative errors in the $L^2$-norm (left) and the $H^1$-norm (right) with respect to the number of parameters for Matérn kernels of smoothness $\nu=3/2$ and fully-connected neural networks in the example with a smooth solution.
  • Figure 3: Relative errors in the $L^2$-norm (left) and the $H^1$-norm (right) with respect to the mesh norm $h$ for Matérn kernels of smoothness $\nu \in \{1/2,3/2,5/2\}$ using the energy minimization problem and the linear system of equations in the example with a smooth solution.
  • Figure 4: Top view plot of the exact solution in the example with a non-convex domain resulting in a singular solution. The singularity occurs in the origin due to steep gradients.
  • Figure 5: Relative errors in the $L^2$-norm (left) and the $H^1$-norm (right) with respect to the mesh norm $h$ for Matérn kernels of smoothness $\nu \in \{1/2,3/2,5/2\}$ in the example with a singular solution. The dashed lines indicate the error decays of the interpolation of the exact solution by the respective kernel. The estimated convergence rate for the interpolation with $\nu=5/2$ is shown as a black line.
  • ...and 2 more figures

Theorems & Definitions (14)

  • Remark 1: Differences to the formulation in e2018deepritz
  • Remark 2: Non-symmetric problems and residual minimization
  • Theorem 1: Céa's Lemma; see cea1964approximation
  • Theorem 2: Aubin-Nitsche Lemma; see ciarlet2002finite
  • Theorem 3: Approximation error estimate for kernel methods in Sobolev norms; see narcowich2004sobolev
  • Theorem 4: A priori error estimate in the $H^1$-norm
  • proof
  • Remark 3: Differences to the a priori error estimate in wendland1999meshless
  • Theorem 5: A priori error estimate in the $L^2$-norm
  • proof
  • ...and 4 more