Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

rKAN: Rational Kolmogorov-Arnold Networks

Alireza Afzal Aghaei

TL;DR

This work introduces rational-function bases for Kolmogorov-Arnold networks (rKANs), presenting two formulations—Padé-based and rational Jacobi function-based—to improve function approximation, especially for features with asymptotic behavior. It treats basis parameters as trainable within the network, enabling end-to-end optimization. Across regression, MNIST classification, and physics-informed problems (Lane-Emden ODE and a elliptic PDE), rKAN variants achieve competitive or superior accuracy in several settings, though Padé-rKAN can incur higher training cost. The study highlights the potential of rational bases to enhance KANs and suggests future exploration of rational B-spline KANs and broader fractional-rKAN evaluations in semi-infinite domains.

Abstract

The development of Kolmogorov-Arnold networks (KANs) marks a significant shift from traditional multi-layer perceptrons in deep learning. Initially, KANs employed B-spline curves as their primary basis function, but their inherent complexity posed implementation challenges. Consequently, researchers have explored alternative basis functions such as Wavelets, Polynomials, and Fractional functions. In this research, we explore the use of rational functions as a novel basis function for KANs. We propose two different approaches based on Pade approximation and rational Jacobi functions as trainable basis functions, establishing the rational KAN (rKAN). We then evaluate rKAN's performance in various deep learning and physics-informed tasks to demonstrate its practicality and effectiveness in function approximation.

rKAN: Rational Kolmogorov-Arnold Networks

TL;DR

This work introduces rational-function bases for Kolmogorov-Arnold networks (rKANs), presenting two formulations—Padé-based and rational Jacobi function-based—to improve function approximation, especially for features with asymptotic behavior. It treats basis parameters as trainable within the network, enabling end-to-end optimization. Across regression, MNIST classification, and physics-informed problems (Lane-Emden ODE and a elliptic PDE), rKAN variants achieve competitive or superior accuracy in several settings, though Padé-rKAN can incur higher training cost. The study highlights the potential of rational bases to enhance KANs and suggests future exploration of rational B-spline KANs and broader fractional-rKAN evaluations in semi-infinite domains.

Abstract

The development of Kolmogorov-Arnold networks (KANs) marks a significant shift from traditional multi-layer perceptrons in deep learning. Initially, KANs employed B-spline curves as their primary basis function, but their inherent complexity posed implementation challenges. Consequently, researchers have explored alternative basis functions such as Wavelets, Polynomials, and Fractional functions. In this research, we explore the use of rational functions as a novel basis function for KANs. We propose two different approaches based on Pade approximation and rational Jacobi functions as trainable basis functions, establishing the rational KAN (rKAN). We then evaluate rKAN's performance in various deep learning and physics-informed tasks to demonstrate its practicality and effectiveness in function approximation.
Paper Structure (13 sections, 1 theorem, 19 equations, 5 figures, 5 tables)

This paper contains 13 sections, 1 theorem, 19 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Jacobi polynomials
  3. Rational KAN
  4. Padé approximation
  5. Rational Jacobi functions
  6. Experiments
  7. Deep learning
  8. Regression Tasks
  9. MNIST classification
  10. Physics-informed Deep Learning
  11. Ordinary Differential Equations
  12. Partial Differential Equations
  13. Conclusion

Key Result

Theorem 3.1

For any continuous function $F : [0,1]^\nu \to \mathbb{R}$, there exist continuous functions $\varphi_{q,k} : [0,1] \to \mathbb{R}$ and continuous functions $\psi_k : \mathbb{R} \to \mathbb{R}$ such that

Figures (5)

  • Figure 1: Plots of mapped Jacobi functions $\mathcal{J}_n^{(0,0)}(\xi)$ for $n=2,3,\ldots,6$ over finite, semi-infinite, and infinite domains.
  • Figure 2: The plots of the functions $F_1(\xi)$, $F_2(\xi)$, and $F_3(\xi)$. The prediction results of rKAN, fKAN, and KAN are presented in Tables \ref{['tbl:f1']}, \ref{['tbl:f2']}, and \ref{['tbl:f3']}.
  • Figure 3: The architecture of proposed method for MNIST classification data.
  • Figure 4: Loss and accuracy of MNIST classification using Jacobi-rKAN with different values of K.
  • Figure 5: The predicted solution and the residual function with respect to the exact solution for the elliptic PDE given in Equation \ref{['eq:pde']}.

Theorems & Definitions (3)

  • Definition 1: Mapped Jacobi function
  • Theorem 3.1
  • proof