Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations

TL;DR

Numerical experiments show that FBPINNs are effective in solving both small and larger, multi-scale problems, outperforming standard PINNs in both accuracy and computational resources required, potentially paving the way to the application of PINNs on large, real-world problems.

Abstract

Recently, physics-informed neural networks (PINNs) have offered a powerful new paradigm for solving problems relating to differential equations. Compared to classical numerical methods PINNs have several advantages, for example their ability to provide mesh-free solutions of differential equations and their ability to carry out forward and inverse modelling within the same optimisation problem. Whilst promising, a key limitation to date is that PINNs have struggled to accurately and efficiently solve problems with large domains and/or multi-scale solutions, which is crucial for their real-world application. Multiple significant and related factors contribute to this issue, including the increasing complexity of the underlying PINN optimisation problem as the problem size grows and the spectral bias of neural networks. In this work we propose a new, scalable approach for solving large problems relating to differential equations called Finite Basis PINNs (FBPINNs). FBPINNs are inspired by classical finite element methods, where the solution of the differential equation is expressed as the sum of a finite set of basis functions with compact support. In FBPINNs neural networks are used to learn these basis functions, which are defined over small, overlapping subdomains. FBINNs are designed to address the spectral bias of neural networks by using separate input normalisation over each subdomain, and reduce the complexity of the underlying optimisation problem by using many smaller neural networks in a parallel divide-and-conquer approach. Our numerical experiments show that FBPINNs are effective in solving both small and larger, multi-scale problems, outperforming standard PINNs in both accuracy and computational resources required, potentially paving the way to the application of PINNs on large, real-world problems.

Figures (13)

• Figure 1: A motivating problem: using PINNs to solve $\frac{d u}{d x} = \cos(\omega x)$. When $\omega=1$ (i.e. $\omega$ is low), a PINN with 2 hidden layers and 16 hidden units is able to rapidly converge to the solution; (a) shows the PINN solution compared to the exact solution and (e) shows the L1 error between the PINN solution and the exact solution against training step. When $\omega=15$ (i.e. $\omega$ is high), the same PINN struggles to converge, as shown in (b). Whilst increasing the size (number of free parameters) of the PINN improves its accuracy, as shown in (c) and (d), it converges much more slowly and with much lower accuracy than the low frequency case, as shown in (e).
• Figure 2: FBPINN workflow. FBPINNs use domain decomposition and separate subdomain normalisation to address the issues related to scaling PINNs to large domains. First, the problem domain is divided into overlapping subdomains; an example 2D hyperrectangular subdivision is shown in (a). Next, separate neural networks are placed within each subdomain. Each network is locally confined to its subdomain by multiplying it with a differentiable window function, such that within the center of the subdomain, the network learns the full solution to the differential equation, whilst in the overlapping regions, the solution is defined as the sum over all overlapping networks. For each network, the input variables $x = (x_1$, $x_2, ...)$ are normalised between [-1,1] over the subdomain, and the output of the network is unnormalised using a common unnormalisation. Finally, an optional constraining operator $\mathcal{C}$ can be applied which appropriately constrains the ansatz such that it automatically satisfies the boundary conditions. FBPINNs are trained using a very similar loss function to standard PINNs which does not require the use of additional interface terms.
• Figure 3: Flexible training schedules for FBPINNs. We can design flexible training schedules which can help to improve the convergence of FBPINNs. These schedules define which subdomain networks are updated during each training step. Within these schedules we define "active" models, which are the networks which are currently being updated, "fixed" models, which are networks which have already been trained and have their free parameters fixed, and "inactive" models which are as-of-yet untrained networks. The plots above show one particular training schedule designed to learn the solution "outwards" from the boundary condition, which in this case is assumed to be along the left edge of the domain. Note during each training step only training points from the active subdomains are required, shown by the coloured regions in the plot.
• Figure 4: Parallel algorithm for training FBPINNs. FBPINNs are trained using gradient descent, and the psuedocode for each training step is shown in (a). The effect of each training step on each subdomain is shown in (b). The algorithm can be implemented using entirely independent threads for each subdomain, except during step 2) where they must share their subdomain network outputs within overlapping regions with the threads of their neighbouring subdomains.
• Figure 5: Performance of FBPINNs on the motivating problem $\frac{d u}{d x} = \cos(\omega x)$ when $\omega=1$. For this case, we find the FPINN described in Section \ref{['sec:1d_low']} has similar performance to the $\omega=1$ PINN in Section \ref{['sec:motivation_low']} (shown in Figure \ref{['fig:cosp']} (a)). The individual FBPINN subdomain solutions after training are shown in (a). The full FBPINN solution compared to the exact solution is shown in (b). The FBPINN subdomain definition, overlapping regions in the domain (thick pink lines), and window function for each subdomain are shown in (c). The L1 error between the FBPINN solution and the exact solution against training step is shown in (d) and (e). Also shown in (d) and (e) are the convergence curves for a FBPINN trained with a smaller subdomain overlap width and the $\omega=1$ PINN from Section \ref{['sec:motivation_low']}.