Solving Partial Differential Equations Using Artificial Neural Networks

TL;DR

This dissertation contributes to the numerical solution of partial differential equations using neural networks with the following two-fold objective: investigate the behavior of neural networks as approximators of solutions of partial differential equations and propose neural-network-based methods for frameworks that are hardly addressable via traditional numerical methods.

Abstract

Partial differential equations have a wide range of applications in modeling multiple physical, biological, or social phenomena. Therefore, we need to approximate the solutions of these equations in computationally feasible terms. Nowadays, among the most popular numerical methods for solving partial differential equations in engineering, we encounter the finite difference and finite element methods. An alternative numerical method that has recently gained popularity for numerically solving partial differential equations is the use of artificial neural networks. Artificial neural networks, or neural networks for short, are mathematical structures with universal approximation properties. In addition, thanks to the extraordinary computational development of the last decade, neural networks have become accessible and powerful numerical methods for engineers and researchers. For example, imaging and language processing are applications of neural networks today that show sublime performance inconceivable years ago. This dissertation contributes to the numerical solution of partial differential equations using neural networks with the following two-fold objective: investigate the behavior of neural networks as approximators of solutions of partial differential equations and propose neural-network-based methods for frameworks that are hardly addressable via traditional numerical methods. As novel neural-network-based proposals, we first present a method inspired by the finite element method when applying mesh refinements to solve parametric problems. Secondly, we propose a general residual minimization scheme based on a generalized version of the Ritz method. Finally, we develop a memory-based strategy to overcome a usual numerical integration limitation when using neural networks to solve partial differential equations.

Figures (43)

• Figure 1: Forward and inverse problems sketch in electromagnetic fields.
• Figure 2: Graph of a fully-connected FFNN with a three-dimensional input, two-dimensional output, depth three, and hidden layers of width five.
• Figure 3: Some typical activation functions appearance: (a) hyperbolic tangent, (b) sigmoid, (c) rectified linear unit, and (d) softplus.
• Figure 4: Sketch of the parameterization of a NN and its objective-function minimization formulation.
• Figure 5: Gradient-descent performance sketch in a convex scenario.

Theorems & Definitions (9)

• Example 3.1: Electromagnetic fields
• Example 4.1: Monte Carlo integration
• Example 4.2: Supervised learning with NN
• Example 4.3
• Example 4.4: Convexity preservation under linear transformation
• proof
• proof
• proof
• Example 6.1: Poisson's Equation