Component Fourier Neural Operator for Singularly Perturbed Differential Equations

TL;DR

Component Fourier Neural Operator (ComFNO), an innovative operator learning method that builds upon Fourier Neural Operator (FNO), while simultaneously incorporating valuable prior knowledge obtained from asymptotic analysis, is introduced.

Abstract

Solving Singularly Perturbed Differential Equations (SPDEs) poses computational challenges arising from the rapid transitions in their solutions within thin regions. The effectiveness of deep learning in addressing differential equations motivates us to employ these methods for solving SPDEs. In this manuscript, we introduce Component Fourier Neural Operator (ComFNO), an innovative operator learning method that builds upon Fourier Neural Operator (FNO), while simultaneously incorporating valuable prior knowledge obtained from asymptotic analysis. Our approach is not limited to FNO and can be applied to other neural network frameworks, such as Deep Operator Network (DeepONet), leading to potential similar SPDEs solvers. Experimental results across diverse classes of SPDEs demonstrate that ComFNO significantly improves accuracy compared to vanilla FNO. Furthermore, ComFNO exhibits natural adaptability to diverse data distributions and performs well in few-shot scenarios, showcasing its excellent generalization ability in practical situations.

Figures (17)

• Figure 1: (left) Ground truth and predictions for FNO and ComFNO. Here we have $b(x)=x+1$, $c(x)=0$ and $\varepsilon=0.001$ in Eq. \ref{['ode_example']}. We take 900 distinct $f(x)$ for training and a random $f(x)$ for testing. (right) Zoomed in curves near the boundary layer. We can see that, the true solution of Eq. \ref{['ode_example']} has a boundary layer at $x=1$.
• Figure 2: Architecture of ComFNO. $a(x)$ and $u(x)$ represent the input of model and solution of the problem, respectively. Both "FNO" and "extra_FNO" represent Fourier Neural Operators, with the latter being smaller. An "exp" function follows "extra_FNO," indicating an exponential operation applied to its output. "Dense" corresponds to a shallow neural network that learns the coefficients of the exponential function. The layer block's input comprises both $a(x)$ and $a(\xi)$, the latter involving a coordinate transformation to accommodate scaling in layers. For instance, when encountering a boundary or inner layer at $x=x_0$, the use of $\xi=(x_0-x)/\varepsilon$ is advantageous.
• Figure 3: Performance of both FNO and ComFNO on Eq.\ref{['eq:ordinary_1']} with $\varepsilon=0.001$. Both trained models are evaluated on 100 $f$ samples, and their resulting residual curves are depicted. (Subfigure): Zoomed-in view.
• Figure 4: FNO and ComFNO performance on Eq.\ref{['eq:ordinary_2']} with $\varepsilon=0.001$. Both trained models are evaluated on 100 $f$ samples, and their resulting residual curves are depicted. (Subfigure): Zoomed-in view.
• Figure 5: Performance of both FNO and ComFNO on Eq.\ref{['eq:partial_1']} with $\varepsilon=0.001$. Both trained models are evaluated on 100 $f$ samples, and their resulting residual curves are depicted. (Subfigure): Zoomed-in view.