HKAN: Hierarchical Kolmogorov-Arnold Network without Backpropagation

TL;DR

The Hierarchical Kolmogorov-Arnold Network is introduced, a novel network architecture that offers a competitive alternative to the recently proposed Kolmogorov-Arnold Network, presenting a robust and efficient alternative for neural network modeling.

Abstract

This paper introduces the Hierarchical Kolmogorov-Arnold Network (HKAN), a novel network architecture that offers a competitive alternative to the recently proposed Kolmogorov-Arnold Network (KAN). Unlike KAN, which relies on backpropagation, HKAN adopts a randomized learning approach, where the parameters of its basis functions are fixed, and linear aggregations are optimized using least-squares regression. HKAN utilizes a hierarchical multi-stacking framework, with each layer refining the predictions from the previous one by solving a series of linear regression problems. This non-iterative training method simplifies computation and eliminates sensitivity to local minima in the loss function. Empirical results show that HKAN delivers comparable, if not superior, accuracy and stability relative to KAN across various regression tasks, while also providing insights into variable importance. The proposed approach seamlessly integrates theoretical insights with practical applications, presenting a robust and efficient alternative for neural network modeling.

Figures (9)

• Figure 1: HKAN architecture.
• Figure 2: Illustration of block function composition using BaFs.
• Figure 3: Synthetic TFs (2D variants of TF4 and TF5).
• Figure 4: Distribution of training (tr) and test (ts) RMSE for KAN, MLP and HKAN.
• Figure 5: Fitted functions and predicted vs. target plots at successive levels of HKAN processing for TF2.