KAN/H: Kolmogorov-Arnold Network using Haar-like bases

TL;DR

KAN/H introduces a Haar-like, hierarchical basis system to replace B-splines in the Kolmogorov-Arnold Network, enabling backpropagation with a real-valued domain and reducing problem-specific hyperparameter tuning. The core idea is the Slash-Haar ($H/$) basis that blends global and local bases via $\Phi_{j,k}$ and $\Psi_{j,k}$, supported by an extended PATRICIA-tree implementation to manage basis coefficients and updates efficiently. The approach is extended to unbounded inputs, and optimization is discussed with Adam and lazy-update strategies, though initial experiments favor SGA. Empirical results on unary-function approximation and MNIST show competitive accuracy with strong robustness to hyperparameters, with CPU-focused demonstrations suggesting substantial potential gains from future GPU acceleration and parallelization.

Abstract

This paper proposes KAN/H, a variant of Kolmogorov-Arnold Network (KAN) that uses a Haar-variant basis system having both global and local bases instead of B-spline. The resulting algorithm is applied to function approximation problems and MNIST. We show that it does not require most of the problem-specific hyper-parameter tunings.

Figures (5)

• Figure 1: Haar basis functions. Each node (j+1,k) has support $[k/2^{j},(k+1)/2^{j}]$, children are obtained by halving the parent's interval.
• Figure 2: Example of (a) visited Haar bases arranged on a binary tree when 3 different samples are input, and (b) its PATRICIA tree representation.
• Figure 3: H / basis system. Each node (j+1,k) has support $[k/2^{j},(k+1)/2^{j}]$, children are obtained by halving the parent's interval.
• Figure 4: Approximation of unary functions using H / basis functions, using the floating-point expressions directly.
• Figure 5: Accuracy of function approximation using the common hyper-parameters, training steps vs. RMSE.