Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PINNIES: An Efficient Physics-Informed Neural Network Framework to Integral Operator Problems

Alireza Afzal Aghaei, Mahdi Movahedian Moghaddam, Kourosh Parand

Abstract

This paper introduces an efficient tensor-vector product technique for the rapid and accurate approximation of integral operators within physics-informed deep learning frameworks. Our approach leverages neural network architectures to evaluate problem dynamics at specific points, while employing Gaussian quadrature formulas to approximate the integral components, even in the presence of infinite domains or singularities. We demonstrate the applicability of this method to both Fredholm and Volterra integral operators, as well as to optimal control problems involving continuous time. Additionally, we outline how this approach can be extended to approximate fractional derivatives and integrals and propose a fast matrix-vector product algorithm for efficiently computing the fractional Caputo derivative. In the numerical section, we conduct comprehensive experiments on forward and inverse problems. For forward problems, we evaluate the performance of our method on over 50 diverse mathematical problems, including multi-dimensional integral equations, systems of integral equations, partial and fractional integro-differential equations, and various optimal control problems in delay, fractional, multi-dimensional, and nonlinear configurations. For inverse problems, we test our approach on several integral equations and fractional integro-differential problems. Finally, we introduce the pinnies Python package to facilitate the implementation and usability of the proposed method.

PINNIES: An Efficient Physics-Informed Neural Network Framework to Integral Operator Problems

Abstract

This paper introduces an efficient tensor-vector product technique for the rapid and accurate approximation of integral operators within physics-informed deep learning frameworks. Our approach leverages neural network architectures to evaluate problem dynamics at specific points, while employing Gaussian quadrature formulas to approximate the integral components, even in the presence of infinite domains or singularities. We demonstrate the applicability of this method to both Fredholm and Volterra integral operators, as well as to optimal control problems involving continuous time. Additionally, we outline how this approach can be extended to approximate fractional derivatives and integrals and propose a fast matrix-vector product algorithm for efficiently computing the fractional Caputo derivative. In the numerical section, we conduct comprehensive experiments on forward and inverse problems. For forward problems, we evaluate the performance of our method on over 50 diverse mathematical problems, including multi-dimensional integral equations, systems of integral equations, partial and fractional integro-differential equations, and various optimal control problems in delay, fractional, multi-dimensional, and nonlinear configurations. For inverse problems, we test our approach on several integral equations and fractional integro-differential problems. Finally, we introduce the pinnies Python package to facilitate the implementation and usability of the proposed method.
Paper Structure (23 sections, 3 theorems, 81 equations, 7 figures, 10 tables)

This paper contains 23 sections, 3 theorems, 81 equations, 7 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Gaussian Quadrature
  3. Jacobi Polynomials
  4. Laguerre Polynomials
  5. Hermite Polynomials
  6. Methodology
  7. Approximating the Solution
  8. Fredholm Integral Operator
  9. Volterra Integral Operator
  10. Multi-dimensional Integral Operators
  11. Fractional Operators
  12. Numerical Results
  13. Sensitivity Analysis and Hyperparameters
  14. One-dimensional Integral Equations
  15. Integro-differntial Integral Equations
  16. ...and 8 more sections

Key Result

Theorem 1

Let $0 < \alpha < 1$ and the interval $[0, x]$ is discretized to $n+1$ points, $0 = x_0 < x_1 < \dots < x_n=x$. Then the following linear combination approximates the Caputo fractional derivative of order $\alpha$: where $u(\cdot)$ is the desired function and $\nu_k$ are real-valued weights.

Figures (7)

  • Figure 1: Diagram illustrating the proposed methodology for solving functional equations that include ordinary, partial, and fractional derivatives, as well as integral operators.
  • Figure 2: A comparison between the Monte Carlo, Newton-Cotes, and Gaussian Quadrature methods for approximating the integral of various functions reveals that Gaussian Quadrature provides greater accuracy with reduced numerical instability. The CPU times for the Monte Carlo, Trapezoid method, Newton-Cotes, and Gauss-Legendre methods with $N=30$ nodal points are $9.5 \pm 0.37$, $19 \pm 0.1$, $8.13 \pm 0.09$, and $8.3 \pm 0.03$ microseconds, respectively. In contrast, computing the derivative of the same function using automatic differentiation takes $21 \pm 0.89$ microseconds. It is evident that the Gauss-Legendre approach is both accurate and efficient.
  • Figure 3: A comparison of four different Volterra and Fredholm integral equations: two with exponential solutions and two with stiff dynamics. The first row illustrates the impact of the number of training points on network accuracy. The middle row examines the effect of varying the number of layers, while the bottom row explores different numbers of hidden neurons in a two-layer neural network. The left column compares CPU time, while the right column evaluates MAE on test data. The reported times represent the mean of five separate runs of the algorithm, using a learning rate of $0.1$ and $50$ epochs. In all cases, we use the hyperbolic tangent function as the nonlinearity mapping.
  • Figure 4: Simulation results of Volterra's population model using the proposed neural network approach for various values of $\kappa$ and different differentiation orders.
  • Figure 5: The trade-off between the residual of the state variable and the system's cost for Example \ref{['ex:opt-delay']}. Increasing the value of $\gamma$ shifts the focus more towards satisfying the constraints while reducing the emphasis on the control variable. This leads to a decrease in the MAE of $\chi(t)$ but results in a higher overall cost. The intersection point of the two line graphs can be considered an optimal value for $\gamma$.
  • ...and 2 more figures

Theorems & Definitions (12)

  • Theorem 1
  • proof
  • Corollary 1
  • Theorem 2
  • proof
  • Example 1
  • Example 2
  • Example 3
  • Example 4
  • Example 5
  • ...and 2 more