Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Deep Uzawa-Lagrange Multiplier Approach for Boundary Conditions in PINNs and Deep Ritz Methods

Charalambos G. Makridakis, Aaron Pim, Tristan Pryer

TL;DR

A deep learning-based framework for weakly enforcing boundary conditions in the numerical approximation of partial differential equations is introduced and the Deep Uzawa algorithm, which incorporates Lagrange multipliers to handle boundary conditions effectively, is proposed.

Abstract

We introduce a deep learning-based framework for weakly enforcing boundary conditions in the numerical approximation of partial differential equations. Building on existing physics-informed neural network and deep Ritz methods, we propose the Deep Uzawa algorithm, which incorporates Lagrange multipliers to handle boundary conditions effectively. This modification requires only a minor computational adjustment but ensures enhanced convergence properties and provably accurate enforcement of boundary conditions, even for singularly perturbed problems. We provide a comprehensive mathematical analysis demonstrating the convergence of the scheme and validate the effectiveness of the Deep Uzawa algorithm through numerical experiments, including high-dimensional, singularly perturbed problems and those posed over non-convex domains.

A Deep Uzawa-Lagrange Multiplier Approach for Boundary Conditions in PINNs and Deep Ritz Methods

TL;DR

A deep learning-based framework for weakly enforcing boundary conditions in the numerical approximation of partial differential equations is introduced and the Deep Uzawa algorithm, which incorporates Lagrange multipliers to handle boundary conditions effectively, is proposed.

Abstract

We introduce a deep learning-based framework for weakly enforcing boundary conditions in the numerical approximation of partial differential equations. Building on existing physics-informed neural network and deep Ritz methods, we propose the Deep Uzawa algorithm, which incorporates Lagrange multipliers to handle boundary conditions effectively. This modification requires only a minor computational adjustment but ensures enhanced convergence properties and provably accurate enforcement of boundary conditions, even for singularly perturbed problems. We provide a comprehensive mathematical analysis demonstrating the convergence of the scheme and validate the effectiveness of the Deep Uzawa algorithm through numerical experiments, including high-dimensional, singularly perturbed problems and those posed over non-convex domains.

Paper Structure

This paper contains 24 sections, 9 theorems, 74 equations, 13 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notation and problem setup
  3. Fractional and Negative Sobolev spaces
  4. Dirichlet energy minimisation
  5. Penalty methods
  6. Proof of Theorem \ref{['thm:Dirichlet_energy_convergence']}
  7. Case 1: $\beta \geq 0$ (Non-negative penalty term)
  8. Case 2: $\beta < 0$ (Negative penalty term)
  9. Least squares minimisation
  10. Proof of Theorem \ref{['thm:PINNs_convergence']}
  11. Neural Networks and Deep Uzawa
  12. Neural Network Function Approximation
  13. Training within the Deep Uzawa Framework
  14. The Deep Uzawa Iteration
  15. Numerical Results
  16. ...and 9 more sections

Key Result

Theorem 2.1

Let $s>\frac{1}{2}$. Then, for $v\in{\operatorname H^{s}}(\Omega)$, there exists a constant $C_{tr} > 0$ such that

Figures (13)

  • Figure 1: Illustration of $\mathcal{C}_L$.
  • Figure 2: Results for Example \ref{['ex:Example_1D_2_BL']} using the RitUz scheme with $\gamma = 2$ and $\epsilon = 10^{-1}$ (left) and $\epsilon = 10^{-3}$ (right). The plots show the state $u_{\theta}$ (top), and the ${\operatorname L}^{2}$-error $\left\|u^k-u^*\right\|_{{\operatorname L}^{2}(\Omega)}$ (bottom) as $\rho$ varies in $[0.01, 20]$.
  • Figure 3: Results for Example \ref{['ex:Example_1D_2_BL']} using the RitUz scheme with $\gamma = 0$ and $\epsilon = 10^{-1}$ (left) and $\epsilon = 10^{-3}$ (right). The plots show the state $u_{\theta}$ (top), and the ${\operatorname L}^{2}$-error $\left\|u^k-u^*\right\|_{{\operatorname L}^{2}(\Omega)}$ (bottom) as $\rho$ varies in $[0.01, 20]$.
  • Figure 4: Results for Example \ref{['ex:Example_1D_2_BL']} using the PINNUz scheme with $\gamma = 2$ and $\epsilon = 10^{-1}$ (left) and $\epsilon = 10^{-3}$ (right). The plots show the state $u_{\theta}$ (top), and the ${\operatorname L}^{2}$-error $\left\|u^k - u^*\right\|_{{\operatorname L}^{2}(\Omega)}$ (bottom) for $\rho \in [10^{-4}, 5]$.
  • Figure 5: Example \ref{['ex:Example_L_shaped']}: (a) Illustration of the exact solution $u^*$ on the L-shaped domain as defined in equation \ref{['eq:L-shaped_exact_soln']}. (b) and (c) show the state $u_\theta$ for $\gamma = 2$, $\epsilon = 10^{-3}$, and $\rho = 1$ and $\rho = 5$, respectively.
  • ...and 8 more figures

Theorems & Definitions (27)

  • Theorem 2.1: Gagliardo57
  • Remark 2.2: Computability of $C_{tr}$
  • Remark 3.1: Finite element approaches
  • Definition 3.2: Dirichlet Loss Functional
  • Remark 3.3: Limitations of penalty only approaches
  • Definition 3.4: Dirichlet energy Lagrangian
  • Definition 3.5: Dirichlet energy update scheme
  • Remark 3.6: Simplifications in the Neural Network Framework
  • Theorem 3.7
  • Remark 3.8: Convergence Regimes
  • ...and 17 more