Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Neural Function Approximation: The XNet Outperforming KAN

Xin Li, Xiaotao Zheng, Zhihong Xia

TL;DR

XNet introduces a single-layer neural network architecture leveraging Cauchy kernels as activations to achieve arbitrarily high-order polynomial convergence, outperforming depth-driven MLPs and B-spline–based KANs in function approximation. The authors provide a theoretical comparison showing $\|f-f_N\|=O(N^{-p})$ for any $p>0$ with Cauchy kernels versus $O(N^{-k})$ for B-splines, and demonstrate substantial empirical gains across function approximation, PINN-based PDE solving, and reinforcement learning. Across Heaviside, high-dimensional, and noisy targets, as well as Heat and Poisson PDEs, XNet achieves dramatically lower errors and faster training than baselines. The results suggest XNet as a versatile, efficient building block for scientific computing and AI applications, with potential for integration into large-scale architectures and time-series models.

Abstract

XNet is a single-layer neural network architecture that leverages Cauchy integral-based activation functions for high-order function approximation. Through theoretical analysis, we show that the Cauchy activation functions used in XNet can achieve arbitrary-order polynomial convergence, fundamentally outperforming traditional MLPs and Kolmogorov-Arnold Networks (KANs) that rely on increased depth or B-spline activations. Our extensive experiments on function approximation, PDE solving, and reinforcement learning demonstrate XNet's superior performance - reducing approximation error by up to 50000 times and accelerating training by up to 10 times compared to existing approaches. These results establish XNet as a highly efficient architecture for both scientific computing and AI applications.

Enhancing Neural Function Approximation: The XNet Outperforming KAN

TL;DR

XNet introduces a single-layer neural network architecture leveraging Cauchy kernels as activations to achieve arbitrarily high-order polynomial convergence, outperforming depth-driven MLPs and B-spline–based KANs in function approximation. The authors provide a theoretical comparison showing for any with Cauchy kernels versus for B-splines, and demonstrate substantial empirical gains across function approximation, PINN-based PDE solving, and reinforcement learning. Across Heaviside, high-dimensional, and noisy targets, as well as Heat and Poisson PDEs, XNet achieves dramatically lower errors and faster training than baselines. The results suggest XNet as a versatile, efficient building block for scientific computing and AI applications, with potential for integration into large-scale architectures and time-series models.

Abstract

XNet is a single-layer neural network architecture that leverages Cauchy integral-based activation functions for high-order function approximation. Through theoretical analysis, we show that the Cauchy activation functions used in XNet can achieve arbitrary-order polynomial convergence, fundamentally outperforming traditional MLPs and Kolmogorov-Arnold Networks (KANs) that rely on increased depth or B-spline activations. Our extensive experiments on function approximation, PDE solving, and reinforcement learning demonstrate XNet's superior performance - reducing approximation error by up to 50000 times and accelerating training by up to 10 times compared to existing approaches. These results establish XNet as a highly efficient architecture for both scientific computing and AI applications.

Paper Structure

This paper contains 30 sections, 1 theorem, 21 equations, 18 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Theoretical Comparison Between Cauchy Kernels and B-splines
  3. Approximation Rate
  4. Computational Efficiency
  5. Numerical Characteristics
  6. Experimental Setup and Results
  7. Function Approximation Experiments
  8. Study Design
  9. Experimental Protocol
  10. Heaviside step function apprxiamtion
  11. Approximation with high-dimensional functions
  12. Functions with noise
  13. PDE Solving Capability
  14. Implementation Details
  15. Heat function
  16. ...and 15 more sections

Key Result

Theorem 1

Cauchy Approximation Theorem (from LXZ24). Let $f(z^1,\dots,z^d)$ be an analytic function on an open set $U \subset \mathbb{C}^d$. It was shown in LXZ24 that for any desired accuracy $\varepsilon > 0$, $f$ can be approximated in the $L^\infty$-norm using a finite sum of Cauchy kernels: Furthermore, the approximation error satisfies $\varepsilon = O(N^{-p})$ for any fixed integer $p$ and sufficien

Figures (18)

  • Figure 1: Heaviside step function approximation comparison: (a) XNet, with 64 basis functions; (b) KAN [1,1], with $k=3$, grid = 200; (c) B-Spline, with $k=3$; (d) KAN [1,1], with $k=3$.
  • Figure 2: Performance of XNet on approximating different functions with varying numbers of parameters: (a) $\exp\left(\frac{1}{2}\left(\sin\left(\pi(x_{1}^{2}+x_{2}^{2})\right) + x_{3}x_{4}\right)\right)$; and (b) $\exp\left(\frac{1}{100}\sum_{i=1}^{100}\sin^2\left(\frac{\pi x_i}{2}\right)\right)$.
  • Figure 3: solution of the Heat equation
  • Figure 4: MLP and KAN Performance
  • Figure 5: XNet Performance
  • ...and 13 more figures

Theorems & Definitions (1)

  • Theorem 1