Element learning: a systematic approach of accelerating finite element-type methods via machine learning, with applications to radiative transfer

TL;DR

A systematic approach for accelerating finite element-type methods by machine learning for the numerical solution of partial differential equations (PDEs) using a neural network to learn the solution map of the PDEs and to do so in an element-wise fashion.

Abstract

In this paper, we propose a systematic approach for accelerating finite element-type methods by machine learning for the numerical solution of partial differential equations (PDEs). The main idea is to use a neural network to learn the solution map of the PDEs and to do so in an element-wise fashion. This map takes input of the element geometry and the PDEs' parameters on that element, and gives output of two operators -- (1) the in2out operator for inter-element communication, and (2) the in2sol operator (Green's function) for element-wise solution recovery. A significant advantage of this approach is that, once trained, this network can be used for the numerical solution of the PDE for any domain geometry and any parameter distribution without retraining. Also, the training is significantly simpler since it is done on the element level instead on the entire domain. We call this approach element learning. This method is closely related to hybridizbale discontinuous Galerkin (HDG) methods in the sense that the local solvers of HDG are replaced by machine learning approaches. Numerical tests are presented for an example PDE, the radiative transfer equation, in a variety of scenarios with idealized or realistic cloud fields, with smooth or sharp gradient in the cloud boundary transition. Under a fixed accuracy level of $10^{-3}$ in the relative $L^2$ error, and polynomial degree $p=6$ in each element, we observe an approximately 5 to 10 times speed-up by element learning compared to a classical finite element-type method.

Figures (17)

• Figure 1: Symbolic representation of the in2out and in2sol operators, from \ref{['eq:in_out']} and \ref{['eq:in_sol']}, on a square element. When these operators from one element are coupled with the operators from neighboring elements, they can be used to find a global solution (see Figure \ref{['fig:hdg_diag']}).
• Figure 2: Solution strategy of both element learning and the standard HDG method. To create a global solver, the local solvers (in2out operators, from Figure \ref{['fig:sch_diagr_in2out_in2sol']}) from neighboring elements are coupled together. In this schematic diagram, we use '$\circ$' for undetermined DOFs and '$\times$' for determined DOFs (solved solution).
• Figure 3: Element learning - architecture. A finite element mesh is broken down element by element. For each element, a neural network takes the element geometry and PDE parameters as inputs, and returns the in2sol and in2out operators, \ref{['eq:in_out']} and \ref{['eq:in_sol']}, as outputs. Then, beyond this schematic diagram, the in2out operators of neighboring elements are systematically coupled together to assemble a global solver (see Figure \ref{['fig:hdg_diag']}).
• Figure 4: A visual representation of the DOFs of the solution $u_h$ and $\widehat{u}_h$. A single $Q_3$ spectral element ($p_x=p_y=3$) is shown with a circle at each of the element's $4\times 4=16$ spatial quadrature points. Each circle also represents the angular coordinate, as $4$ uniformly-partitioned $P_0$ angular elements ($N_a=4$ and $p_a=0$) at each spatial quadrature point. On the skeleton along the boundary of the spatial element, the letters 'i' and 'o' represent the inflow and the outflow DOFs of $\widehat{u}_h$, respectively.
• Figure 5: Training the neural network. Left: $9$ samples of the scattering coefficient $\sigma_s$ generated from the Algorithm \ref{['algrm:opt_data_gen']}. Right: testing error for the networks with different number of layers. The testing error is lowest (and similar) for 3 or 4 layers. The neural network is able to learn the nonlinear operator from \ref{['eq:param2sol']} with an error that is nearly as small as $10^{-4}$.