Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Neural Networks and Finite Elements of Any Order on Arbitrary Dimensions

Juncai He, Jinchao Xu

TL;DR

This work proves that deep neural networks with ReLU and ReLU^2 activations can exactly represent Lagrange finite element functions of any order on arbitrary simplicial meshes in any dimension by introducing two global basis formulations based on a geometric decomposition of interpolation points and barycentric coordinates. It develops a concrete bridge between finite element basis theory and neural network expressivity, providing explicit DNN architectures, depth, and width bounds that realize the basis functions for both vertex-convex and arbitrary meshes using the mixed activation class $\Sigma^{1,2}_{n_{1:L}}$. The results yield a direct representation of any Lagrange finite element function and an accompanying $W^{s,p}$ approximation theory on uniform and general meshes, with practical implications for FE implementations and potential adaptive strategies in PDE solving. Overall, the paper establishes a principled, scalable framework to encode finite element spaces within DNNs, enabling systematic neural representations of high-order, multi-dimensional piecewise polynomials.

Abstract

In this study, we establish that deep neural networks employing ReLU and ReLU$^2$ activation functions can effectively represent Lagrange finite element functions of any order on various simplicial meshes in arbitrary dimensions. We introduce two novel formulations for globally expressing the basis functions of Lagrange elements, tailored for both specific and arbitrary meshes. These formulations are based on a geometric decomposition of the elements, incorporating several insightful and essential properties of high-dimensional simplicial meshes, barycentric coordinate functions, and global basis functions of linear elements. This representation theory facilitates a natural approximation result for such deep neural networks. Our findings present the first demonstration of how deep neural networks can systematically generate general continuous piecewise polynomial functions on both specific or arbitrary simplicial meshes.

Deep Neural Networks and Finite Elements of Any Order on Arbitrary Dimensions

TL;DR

This work proves that deep neural networks with ReLU and ReLU^2 activations can exactly represent Lagrange finite element functions of any order on arbitrary simplicial meshes in any dimension by introducing two global basis formulations based on a geometric decomposition of interpolation points and barycentric coordinates. It develops a concrete bridge between finite element basis theory and neural network expressivity, providing explicit DNN architectures, depth, and width bounds that realize the basis functions for both vertex-convex and arbitrary meshes using the mixed activation class . The results yield a direct representation of any Lagrange finite element function and an accompanying approximation theory on uniform and general meshes, with practical implications for FE implementations and potential adaptive strategies in PDE solving. Overall, the paper establishes a principled, scalable framework to encode finite element spaces within DNNs, enabling systematic neural representations of high-order, multi-dimensional piecewise polynomials.

Abstract

In this study, we establish that deep neural networks employing ReLU and ReLU activation functions can effectively represent Lagrange finite element functions of any order on various simplicial meshes in arbitrary dimensions. We introduce two novel formulations for globally expressing the basis functions of Lagrange elements, tailored for both specific and arbitrary meshes. These formulations are based on a geometric decomposition of the elements, incorporating several insightful and essential properties of high-dimensional simplicial meshes, barycentric coordinate functions, and global basis functions of linear elements. This representation theory facilitates a natural approximation result for such deep neural networks. Our findings present the first demonstration of how deep neural networks can systematically generate general continuous piecewise polynomial functions on both specific or arbitrary simplicial meshes.
Paper Structure (19 sections, 13 theorems, 82 equations, 3 figures)

This paper contains 19 sections, 13 theorems, 82 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Simplex, sub-simplex, simplicial mesh, and their properties in $\mathbb R^d$.
  4. Lagrange finite elements and global basis functions in classical definition.
  5. Geometric Decomposition of $\mathcal{X}_k({\mathcal{T}})$.
  6. New formulations of global Lagrange element basis functions of any order on any dimensions
  7. A novel formulation for global basis functions on vertex-convex meshes.
  8. Proof of Theorem \ref{['thm:basisall-convex']}
  9. A novel formulation for global basis functions on arbitrary simplicial meshes.
  10. Proof of Theorem \ref{['thm:basisall-noncovex']}
  11. Representations of Lagrange elements using DNNs with ReLU and ReLU$^2$ activation functions
  12. Representation of basis functions on vertex-convex meshes.
  13. Representation of basis functions on arbitrary simplicial meshes.
  14. Representation of Lagrange finite element functions using $\Sigma_{n_{1:L}}^{1,2}$.
  15. Proof of Themorem \ref{['thm:uhdnn']}
  16. ...and 4 more sections

Key Result

Theorem 2.1

For any $\bm p = \bm x_{\bm \alpha} \in \mathcal{X}_k(T)$ with $\bm \alpha \in \mathbb T_k^d$ and $\bm{x} \in T$, the basis function is given by where $\bm \alpha ! = \prod_{i=0}^d \left(\alpha_i !\right)$.

Figures (3)

  • Figure 2.1: An example of the decomposition of $\mathcal{X}_4(T)$ on a 2D simplex $T$ is illustrated. Here, $\mathcal{X}_4^0(T)$ denotes all the vertices of $T$ (marked in blue), $\mathcal{X}_4^1(T)$ represents all nodes on the interiors of edges (marked in green), and $\mathcal{X}_4^2(T)$ includes all nodes in the interior of the simplex $T$ (marked in yellow).
  • Figure 3.1: An example of $\bm p \in \mathcal{X}_2^1(\mathcal{T})$ on 2D, where $I_{\bm p} = \{s ,s'\}$ and $\bm p \in f = {\rm Conv}(\bm v_0^f, \bm v_1^f)$. We have $\min\left\{\lambda_{s,i}(\bm x), \lambda_{s',i}(\bm x)\right\} = \lambda_{s,i}(\bm x)$ for any $\bm x \in T_s$ and $i=0,1$.
  • Figure 3.2: An example on 2D showing that $\mathcal{S}'$ is nonempty. In this example, $P_{\bm v_0^f}$ and $P_{\bm v_1^f}$ are not convex, and $S' = f' \in \Delta_1(\mathcal{T})$ is a 1-dimensional sub-simplex marked as the red segment in the diagram. Here, we have $f' \subseteq \partial P_{\bm v_0^f} \cap \partial P_{\bm v_1^f}$.

Theorems & Definitions (24)

  • Theorem 2.1: burkardt2013finitechen2023geometric
  • Definition 2.1
  • Definition 3.1
  • Definition 3.2
  • Lemma 3.1
  • Theorem 3.1
  • Lemma 3.2
  • Lemma 3.3
  • Remark 1
  • Lemma 3.4
  • ...and 14 more