Exploring the Limitations of Kolmogorov-Arnold Networks in Classification: Insights to Software Training and Hardware Implementation

TL;DR

This paper evaluates Kolmogorov-Arnold Networks (KANs) for classification and hardware deployment. It compares KANs against MLPs on four datasets and implements hardware with Vitis HLS on an FPGA, showing that KANs do not outperform MLPs in accuracy and incur higher hardware costs; symbolic conversion further degrades performance. The work formalizes KANs via the Kolmogorov-Arnold representation f(x_1,...,x_n) = sum_{q=1}^{2n+1} Phi_q(sum_{p=1}^n phi_{q,p}(x_p)) and models the network as KAN(x) = (Phi_{L-1} o ... o Phi_0) x, highlighting practical trade-offs between accuracy and resources. The findings argue for MLPs in practical software and hardware contexts and suggest directions to optimize KAN architectures and tooling.

Abstract

Kolmogorov-Arnold Networks (KANs), a novel type of neural network, have recently gained popularity and attention due to the ability to substitute multi-layer perceptions (MLPs) in artificial intelligence (AI) with higher accuracy and interoperability. However, KAN assessment is still limited and cannot provide an in-depth analysis of a specific domain. Furthermore, no study has been conducted on the implementation of KANs in hardware design, which would directly demonstrate whether KANs are truly superior to MLPs in practical applications. As a result, in this paper, we focus on verifying KANs for classification issues, which are a common but significant topic in AI using four different types of datasets. Furthermore, the corresponding hardware implementation is considered using the Vitis high-level synthesis (HLS) tool. To the best of our knowledge, this is the first article to implement hardware for KAN. The results indicate that KANs cannot achieve more accuracy than MLPs in high complex datasets while utilizing substantially higher hardware resources. Therefore, MLP remains an effective approach for achieving accuracy and efficiency in software and hardware implementation.

Figures (3)

• Figure 1: The generalized KAN architecture
• Figure 2: The training time between MLPs and KANs
• Figure 3: The loss during training processes of MLPs and KANs