X-KAN: Optimizing Local Kolmogorov-Arnold Networks via Evolutionary Rule-Based Machine Learning

TL;DR

X-KAN addresses the challenge of approximating functions with local nonlinearities and discontinuities by deploying multiple local Kolmogorov-Arnold Networks (KANs) within an XCSF-inspired rule-based framework. It jointly optimizes rule antecedents via evolutionary search and local KAN parameters via backpropagation, producing compact, generalizable local models as IF-THEN rules. Empirical results on eight problems show that X-KAN consistently outperforms XCSF, MLP, and a global KAN inMAE, with an average of 7.2 rules in the compacted set, and analysis confirms the importance of balancing accuracy with generality. The work demonstrates a principled divide-and-conquer approach to function approximation that leverages KAN’s expressiveness while mitigating global-model limitations, with potential extensions to interpretable piecewise regression.

Abstract

Function approximation is a critical task in various fields. However, existing neural network approaches struggle with locally complex or discontinuous functions due to their reliance on a single global model covering the entire problem space. We propose X-KAN, a novel method that optimizes multiple local Kolmogorov-Arnold Networks (KANs) through an evolutionary rule-based machine learning framework called XCSF. X-KAN combines KAN's high expressiveness with XCSF's adaptive partitioning capability by implementing local KAN models as rule consequents and defining local regions via rule antecedents. Our experimental results on artificial test functions and real-world datasets demonstrate that X-KAN significantly outperforms conventional methods, including XCSF, Multi-Layer Perceptron, and KAN, in terms of approximation accuracy. Notably, X-KAN effectively handles functions with locally complex or discontinuous structures that are challenging for conventional KAN, using a compact set of rules (average 7.2 $\pm$ 2.3 rules). These results validate the effectiveness of using KAN as a local model in XCSF, which evaluates the rule fitness based on both accuracy and generality. Our X-KAN implementation is available at https://github.com/YNU-NakataLab/X-KAN.

Figures (11)

• Figure 1: An example of a ruleset of X-KAN with three rules, $k_1, k_2$, and $k_3$, in an input space $[0,1]^2$. X-KAN partitions the input space into local hyperrectangular regions defined by rule antecedents and performs local function approximation within each region using a KAN model implemented in the rule consequent. Appendix \ref{['sec: sup the architecture of X-KAN']} schematically illustrates the architecture of X-KAN.
• Figure 2: The two-dimensional test functions $f_1$-$f_4$. (a) $f_1$ has high curvature and non-linearity in both dimensions. (b) $f_2$ features intricate nonlinear patterns and multiple local extrema. (c) $f_3$ combines both linear and nonlinear local regions. (d) $f_4$ is characterized by relatively smooth regions in its central domain but exhibits steep gradients near its boundaries.
• Figure 5: Comparison of fitness functions: X-KAN (accuracy and generality) vs. X-KAN$\kappa$ (accuracy only). Notation follows Table \ref{['tb:result']}.
• Figure 6: Average runtime per trial. Extensions .jl/.py indicate implementations in Julia/Python.
• Figure 7: Comparison of MLPs and KANs architectures, both consisting of one input layer, one hidden layer, and one output layer, with input dimension $n=2$. MLPs employ the SiLU function as their activation function. For simplicity of visualization, bias terms in MLPs architectures have been omitted.