Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sinc Kolmogorov-Arnold Network and Its Applications on Physics-informed Neural Networks

Tianchi Yu, Jingwei Qiu, Jiang Yang, Ivan Oseledets

TL;DR

This paper proposes to use Sinc interpolation in the context of Kolmogorov-Arnold Networks, neural networks with learnable activation functions, and shows that SincKANs provide better results in almost all of the examples.

Abstract

In this paper, we propose to use Sinc interpolation in the context of Kolmogorov-Arnold Networks, neural networks with learnable activation functions, which recently gained attention as alternatives to multilayer perceptron. Many different function representations have already been tried, but we show that Sinc interpolation proposes a viable alternative, since it is known in numerical analysis to represent well both smooth functions and functions with singularities. This is important not only for function approximation but also for the solutions of partial differential equations with physics-informed neural networks. Through a series of experiments, we show that SincKANs provide better results in almost all of the examples we have considered.

Sinc Kolmogorov-Arnold Network and Its Applications on Physics-informed Neural Networks

TL;DR

This paper proposes to use Sinc interpolation in the context of Kolmogorov-Arnold Networks, neural networks with learnable activation functions, and shows that SincKANs provide better results in almost all of the examples.

Abstract

In this paper, we propose to use Sinc interpolation in the context of Kolmogorov-Arnold Networks, neural networks with learnable activation functions, which recently gained attention as alternatives to multilayer perceptron. Many different function representations have already been tried, but we show that Sinc interpolation proposes a viable alternative, since it is known in numerical analysis to represent well both smooth functions and functions with singularities. This is important not only for function approximation but also for the solutions of partial differential equations with physics-informed neural networks. Through a series of experiments, we show that SincKANs provide better results in almost all of the examples we have considered.
Paper Structure (45 sections, 2 theorems, 40 equations, 9 figures, 10 tables)

This paper contains 45 sections, 2 theorems, 40 equations, 9 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Physics-informed neural networks (PINNs)
  4. Sinc numerical methods
  5. Sinc Kolmogorov-Arnold Network (SincKAN)
  6. Optimal $h$.
  7. Coordinate transformation.
  8. Experiments
  9. Learning for approximation
  10. Selecting h
  11. Relationship between degree and size of data
  12. Learning for PIKANs
  13. Boundary layer problem
  14. Ablation study
  15. Conclusion
  16. ...and 30 more sections

Key Result

Theorem 1

sugihara2004recent Assume $\alpha, \beta, d >0$, that (1) $f$ belongs to $H^1\left(\mathcal{D}_d\right)$, where $H^1$ is the Hardy space and $\mathcal{D}_d=\{z\in \mathbb{C} \ | \ \left|\Im z\right|<d\}$; (2) $f$ decays exponentially on the real line, that is, $|f(x)| \leq \alpha \exp (-\beta|x|), \ for some constant $C$, where the step size $h$ is taken as

Figures (9)

  • Figure 1: \ref{['sqrt_matlab']} depicts the Sinc's merit of handling the end-point singularity while the Chebyshev and the spline converge slowly. \ref{['bl_matlab']} shows that, for the boundary layer functions that have high derivatives, Sinc converges exponentially while Chebyshev converges slowly at first. \ref{['bl10000_solution']} partially depicts the solution and the interpolations over the interval $[0,0.08]$, indicating that Sinc interpolation provides the most accurate approximation, while Chebyshev interpolation exhibits significant oscillations, and spline interpolation shows localized inaccuracies in certain regions
  • Figure 2: \ref{['piecewise_solution']} depicts the function of 'piece-wise' in \ref{['table: function approxmation']} and compares the performance of SincKAN with MLP and KAN. \ref{['piecewise_error']} demonstrate the convergence of relative error for all networks, note that although the ChebyKAN we used is the modified ChebyKAN, its training is still unstable. Herein, the results of ChebyKAN in this paper are always the last valid error. \ref{['piecewise_sub1']} and \ref{['piecewise_sub2']} demonstrate the singularities in detail and show that the SincKAN can approximate the singularities well while MLP and KAN have obvious differences.
  • Figure 3: \ref{['sin_low_inverse']} shows the inverse decay approach on sin-low function; \ref{['sin_high_inverse']} shows the inverse decay approach on sin-high function.
  • Figure 4: Demonstration of boundary layer problems, \ref{['bl_solution_sinckan']} depicts the exact solution of \ref{['eq: bl']} with different $\epsilon$ and the corresponding predicted solution by SincKAN, states that SincKAN can solve \ref{['eq: bl']} properly even with an extremely narrow boundary layer. \ref{['bl_loss_sinckan']} depicts the convergence of the training loss function of SincKAN with different $\epsilon$. \ref{['bl_1000_sinckan']} demonstrates the results of different networks when solving \ref{['eq: bl']} with $\epsilon=1000$ and reveals that the derivative in the boundary layer is so large that other networks cannot approximate the boundary layer well.
  • Figure 5: \ref{['bl_2d_test']} depicts the exact solution of \ref{['eq:bl2d']}, \ref{['bl_2d_pred']} shows the solution predicted by SincKAN, \ref{['bl_2d_error']} shows the absolute error between the predicted solution and the exact solution, exhibits that the error mainly comes from the boundary layer.
  • ...and 4 more figures

Theorems & Definitions (2)

  • Theorem 1
  • Theorem 2