Learning Contractive Integral Operators with Fredholm Integral Neural Operators

Abstract

We generalize the framework of Fredholm Neural Networks, to learn non-expansive integral operators arising in Fredholm Integral Equations (FIEs) of the second kind in arbitrary dimensions. We first present the proposed Fredholm Integral Neural Operators (FREDINOs), for FIEs and prove that they are universal approximators of linear and non-linear integral operators and corresponding solution operators. We furthermore prove that the learned operators are guaranteed to be contractive, thereby strictly satisfying the mathematical property required for the convergence of the fixed point scheme. Finally, we also demonstrate how FREDINOs can be used to learn the solution operator of non-linear elliptic PDEs, via a Boundary Integral Equation (BIE) formulation. We assess the proposed methodology numerically, via several benchmark problems: linear and non-linear FIEs in arbitrary dimensions, as well as a non-linear elliptic PDE in 2D. Built on tailored mathematical/numerical analysis theory, FREDINOs offer high-accuracy approximations and interpretable schemes, making them well suited for scientific machine learning/numerical analysis computations.

Figures (8)

• Figure 1: Schematic of: (Left) The Feedfroward Neural Network (Right) The Recurrent Fredholm Neural Network. The recurrent layer applies the non-linearity $G(\cdot)$ pointwise to the output of each node. The grid points $z_i \in \mathbb{R}^d$ can be sampled either uniformly or using quasi-Monte Carlo methods such as Sobol sampling.
• Figure 2: The Fredholm Neural Operator (FREDINO) framework: The training data consists of a family of functions $g_i$ and the corresponding solutions to the FIE $f_i$, for $i=1,\dots, n$. We model the unknown integral kernel and the non-linearity with two neural networks $K_{\theta}(x,y)$ and $G_{\vartheta}(x)$. These models then populate the parameters of the Fredholm Neural Network. The loss function $\mathcal{L}(\Theta)$ is calculated by comparing output of the populated Fredholm NN, $\hat{f}_i, i=1,\dots,n$ with the training data. For brevity, the schematic refers to the case of the non-linear FIE, using the Recurrent Fredholm NN. The linear case is simpler, where we only consider $K_{\theta}$.
• Figure 3: The Potential Fredholm Neural Network (PFNN) for the solution of 2D linear elliptic PDEs: the first component is a Fredholm NN solving the Boundary Integral Equation and the final hidden layer corresponds to the double layer potential formulation of the solution to the PDE, $\tilde{u}(r, \phi)$.
• Figure 4: Results for Example \ref{['ex-linear']}: (Top left) The loss function evolution during training. (Top right) Comparison between the true solution to the integral equation $f_{test}$, for a given $g_{test}$ in the test set, and the estimated solution $\hat{f}_{test}(x)$, as calculated by the forward Fredholm NN with the best (minimum rel. $L_2$ error) learned kernel ${K}_{\theta}$. (Bottom left) The sup-norm of the consecutive layers in the Fredholm NN with the learned kernel, showing that the learned integral operator is a contraction and converges to the fixed point solution. (Right) The contour of the learned kernel model $K_{\theta}.$
• Figure 5: Results for Example \ref{['high-d-example']}: (Top left) The loss function evolution during training. (Top middle) Comparison between the true solution to the integral equation $f$ in the test set, and the estimated solution $\hat{f}(x)$, as calculated by the forward Fredholm NN with the best (minimum rel. $L_2$ error) learned kernel ${K}_{\theta}$. For the illustration we plot the values at each grid node, with the nodes reordered so that $f(x)$ increases monotonically. (Top right) The plot of the corresponding absolute errors, $|\hat{f}(x) - f(x)|$, at each grid node. (Bottom left) The distribution of the absolute errors for the test functions $g^{test}$, across all $N=1024$ Sobol samples in $\mathbb{R}^{10}$. (Bottom middle) The norm of the consecutive layers in the Fredholm NN with the learned kernel, showing the convergence to the fixed point solution. (Bottom right) The contour of a slice of the learned kernel model $K_{\theta}(x, z)$ with $x = (0.5, \cdots, 0.5)$ and $z = (z_1, 0.5, \dots, 0.5, z_{10})$.

Theorems & Definitions (19)

• Remark 2.1
• Definition 2.2: Contractive operator
• Proposition 2.3: Krasnosel'skii-Mann (KM) method
• Definition 2.4: Discretized Krasnoselskii-Mann operator
• Remark 2.5
• Definition 2.6: $M-$layer fixed point estimate
• Proposition 2.7: Fredholm Neural Network construction
• Theorem 2.8: Fredholm NNs as universal approximators of linear FIE solutions georgiou2025fredholm
• Proposition 2.9: Recurrent Fredholm NN construction
• Theorem 2.10: Fredholm NNs as universal approximators of non-linear FIE solutions