Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fredholm Neural Networks

Kyriakos Georgiou, Constantinos Siettos, Athanasios N. Yannacopoulos

TL;DR

The paper introduces Fredholm neural networks, architectures that encode fixed-point iterations for linear and nonlinear Fredholm integral equations into a deep neural network structure. By discretizing the integral operator and mapping each iteration to a network layer, they achieve an explainable framework with convergence guarantees and explicit error bounds, extending to inverse problems via kernel learning. The authors demonstrate strong numerical accuracy across linear, nonlinear, boundary-value, and elliptic PDE scenarios, with notably superior boundary handling compared to traditional methods. They propose this approach as a principled alternative to black-box surrogate networks, with potential impact on uncertainty quantification and scientific computing.

Abstract

Within the family of explainable machine-learning, we present Fredholm neural networks (Fredholm NNs): deep neural networks (DNNs) architectures motivated by fixed-point iteration schemes for the solution of linear and nonlinear Fredholm integral equations (FIEs) of the second kind. We also show how the proposed framework can be used for the solution of inverse problems. Applications of FIEs include the solution of ordinary, as well as partial differential equations (ODEs, PDEs) and many more. We first prove that Fredholm NNs provide accurate solutions. We then provide insight into the values of the hyperparameters and trainable/explainable weights and biases of the DNN, by directly connecting their values to the underlying mathematical theory. For our illustrations, we use Fredholm NNs to solve both linear and nonlinear problems, including elliptic PDEs and boundary value problems. We show that the proposed scheme achieves significant numerical approximation accuracy across both the domain and boundary. The proposed methodology provides insight into the connection between neural networks and classical numerical methods, and we posit that it can have applications in fields such as Uncertainty Quantification (UQ) and explainable artificial intelligence (XAI). Thus, we believe that it will trigger further advances in the intersection between scientific machine learning and numerical analysis.

Fredholm Neural Networks

TL;DR

The paper introduces Fredholm neural networks, architectures that encode fixed-point iterations for linear and nonlinear Fredholm integral equations into a deep neural network structure. By discretizing the integral operator and mapping each iteration to a network layer, they achieve an explainable framework with convergence guarantees and explicit error bounds, extending to inverse problems via kernel learning. The authors demonstrate strong numerical accuracy across linear, nonlinear, boundary-value, and elliptic PDE scenarios, with notably superior boundary handling compared to traditional methods. They propose this approach as a principled alternative to black-box surrogate networks, with potential impact on uncertainty quantification and scientific computing.

Abstract

Within the family of explainable machine-learning, we present Fredholm neural networks (Fredholm NNs): deep neural networks (DNNs) architectures motivated by fixed-point iteration schemes for the solution of linear and nonlinear Fredholm integral equations (FIEs) of the second kind. We also show how the proposed framework can be used for the solution of inverse problems. Applications of FIEs include the solution of ordinary, as well as partial differential equations (ODEs, PDEs) and many more. We first prove that Fredholm NNs provide accurate solutions. We then provide insight into the values of the hyperparameters and trainable/explainable weights and biases of the DNN, by directly connecting their values to the underlying mathematical theory. For our illustrations, we use Fredholm NNs to solve both linear and nonlinear problems, including elliptic PDEs and boundary value problems. We show that the proposed scheme achieves significant numerical approximation accuracy across both the domain and boundary. The proposed methodology provides insight into the connection between neural networks and classical numerical methods, and we posit that it can have applications in fields such as Uncertainty Quantification (UQ) and explainable artificial intelligence (XAI). Thus, we believe that it will trigger further advances in the intersection between scientific machine learning and numerical analysis.
Paper Structure (12 sections, 9 theorems, 63 equations, 13 figures, 2 algorithms)

This paper contains 12 sections, 9 theorems, 63 equations, 13 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Methodology
  3. Solving Fredholm Integral Equations
  4. Connection between linear FIEs and Deep Neural Networks
  5. Extension to non-linear FIEs
  6. Implementation and examples
  7. Linear Fredholm Integral Equations
  8. Boundary Value problems
  9. Non-linear Fredholm Integral Equations
  10. Application to elliptic PDEs: Boundary Integrals
  11. Fredholm Neural Networks in Inverse Problems
  12. Conclusions and Discussion

Key Result

Proposition 2.3

Consider the linear FIE (ie) defined by a non-expansive operator $\mathcal{T}$, on ${\cal H}$. Furthermore, consider a sequence $\{\kappa_n\}, \kappa_n \in (0,1]$ such that $\sum_n \kappa_n(1-\kappa_n) = \infty$. Then the iterative scheme: with $f_0(x) = g(x)$, converges to the fixed point solution of the FIE, $f^{*}(x)$.

Figures (13)

  • Figure 1: Schematic of the proposed Fredholm NN as a fixed-point DNN with $M=3$ hidden layers; each node corresponds to a value on a discretized grid (we consider a constant value of the parameter $\kappa$): (a) One-dimensional output, (b) Multi-dimensional output across a grid.
  • Figure 2: Full Fredholm NN model for estimating the solution of FIEs. The first component solves the IE along the pre-defined grid, followed by the last layer to obtain the final output.
  • Figure 3: The recurrent Fredholm NNs as given in algorithm \ref{['alg:nl-fnn']}, for the solution of non-linear FIEs. We used the Fredholm NN to solve the FIE (\ref{['nl-iteration']}) at each iteration and used the result to re-define the additive term in the linear FIE.
  • Figure 4: Results for the example \ref{['ex-1']}: (a) Exact solution and Fredholm NN approximation with 10 layers, (b) Maximum error as a function of the number of hidden layers.
  • Figure 5: Results for the example \ref{['ex-2']}: (a) Exact solution and Fredholm NN approximation with 15 layers (b) Error in the Fredholm NN (c) Maximum error with respect to the number of hidden layers.
  • ...and 8 more figures

Theorems & Definitions (29)

  • Definition 2.2
  • Proposition 2.3: Krasnoselskii-Mann (KM) method
  • Definition 2.4: $M-$layer fixed point estimate
  • Remark 2.5
  • Lemma 2.6: Fredholm Neural Network
  • proof
  • Theorem 2.7
  • proof
  • Remark 2.8
  • Proposition 2.9
  • ...and 19 more