Deep Ritz-Finite Element methods: Neural Network Methods trained with Finite Elements

TL;DR

This work addresses solving linear elliptic PDEs on domains $\Omega$ by embedding neural-network trial spaces into a variational Deep Ritz framework and computing the energy loss via finite-element quadrature. It introduces three computable training variants—quadrature/collocation, Monte-Carlo quadrature, and a Finite-Element–based energy—melding neural networks with FE tooling to achieve stable, convergent approximations to the PDE solution. The authors prove stability (equi-coercivity) and convergence of discrete minimisers using a $\Gamma$-convergence–style analysis and provide numerical evidence showing that FE-based training outperforms standard NN collocation methods in accuracy and computational efficiency. The results offer a practical, hybrid approach that leverages FE methods within neural-network solvers for elliptic problems, with implications for integrating adaptivity and mesh generation in NN-PDE solvers.

Abstract

While much attention of neural network methods is devoted to high-dimensional PDE problems, in this work we consider methods designed to work for elliptic problems on domains $Ω\subset \mathbb{R} ^d, $ $d=1,2,3$ in association with more standard finite elements. We suggest to connect finite elements and neural network approximations through training, i.e., using finite element spaces to compute the integrals appearing in the loss functionals. This approach, retains the simplicity of classical neural network methods for PDEs, uses well established finite element tools (and software) to compute the integrals involved and it gains in efficiency and accuracy. We demonstrate that the proposed methods are stable and furthermore, we establish that the resulting approximations converge to the solutions of the PDE. Numerical results indicating the efficiency and robustness of the proposed algorithms are presented.

Figures (8)

• Figure 3.1: Uniform mesh: The unit square is divided in $2 M^2$ triangles (cells), here $M=5$. In the simulation that follow $M=$ 20, 40, 60, 80, 100, 120.
• Figure 3.2: A Residual Neural Network.
• Figure 3.3: Monte-Carlo Collocation. Energy minimization through collocation points varying collocation points and the blocks number of the Residual Network architecture. Top image: For a given number of collocation points $\lVert u_\theta - u_e\rVert_{L^2(\varOmega)}$ is computed where $u_\theta, u_e$ denote the discrete, exact minimizers respectively. The minimum error $1.9 \cdot 10^{-2}$ is achieved for $1$ block and $M=80$, i.e. 13120 collocation points. Bottom image: Solid curves illustrate loss function values after the final epoch iteration (left vertical axis) and dashed curves the total execution time (right vertical axis).
• Figure 3.4: Quadrature collocation. Energy minimization through quadrature rule with degree of precision 1. The number of cells and the blocks number of the Residual Network are varied. Top image: For a given number of cells $\lVert u_\theta - u_e\rVert_{L^2(\varOmega)}$ is computed. The minimum error $1.6 \cdot 10^{-3}$ is achieved for $3$ blocks and $M=100$, i.e. 20000 triangles. Bottom image: Solid curves illustrate loss function values after the final epoch iteration (left vertical axis) and dashed curves the total execution time (right vertical axis).
• Figure 3.5: Finite Element training. Energy minimization through quadrature rule, with degree of precision 1, and finite element interpolation. The number of cells and the blocks number of the Residual Network are varied. Top image: For a given number of cells $\lVert u_\theta - u_e\rVert_{L^2(\varOmega)}$ is computed. The minimum error $7.2 \cdot 10^{-4}$ is achieved for $4$ blocks and $M=120$, i.e. 28800 triangles. Bottom image: Solid curves illustrate loss function values after the final epoch iteration (left vertical axis) and dashed curves the total execution time (right vertical axis).

Theorems & Definitions (10)

• Definition 1.1
• Definition 1.2
• Definition 1.3
• Remark 2.1
• Proposition 2.1
• proof
• Theorem 2.1: Convergence of the discrete minimisers
• proof
• Remark 2.2
• Remark 2.3: Convergence for General Elliptic Problems