FC-KAN: Function Combinations in Kolmogorov-Arnold Networks

TL;DR

The paper addresses representing complex multivariate mappings by leveraging function combinations within Kolmogorov-Arnold Networks (KANs). It introduces FC-KAN, which applies multiple basis-function families (e.g., B-splines, DoG, GRBFs) to low-dimensional outputs and combines them via element-wise, concatenation, or linearization, with a quadratic output representation guiding the combinations. The approach is evaluated on MNIST and Fashion-MNIST, where FC-KAN variants, particularly DoG+BS and BS+BASE, achieve top performance among KANs while incurring higher training time due to the quadratic combination, and misclassification analyses show generally fewer errors per class. The work demonstrates the practical potential of function combinations in KANs and provides a public implementation, while outlining limitations and avenues for future exploration such as deeper integration across layers and broader dataset validation.

Abstract

In this paper, we introduce FC-KAN, a Kolmogorov-Arnold Network (KAN) that leverages combinations of popular mathematical functions such as B-splines, wavelets, and radial basis functions on low-dimensional data through element-wise operations. We explore several methods for combining the outputs of these functions, including sum, element-wise product, the addition of sum and element-wise product, representations of quadratic and cubic functions, concatenation, linear transformation of the concatenated output, and others. In our experiments, we compare FC-KAN with a multi-layer perceptron network (MLP) and other existing KANs, such as BSRBF-KAN, EfficientKAN, FastKAN, and FasterKAN, on the MNIST and Fashion-MNIST datasets. Two variants of FC-KAN, which use a combination of outputs from B-splines and Difference of Gaussians (DoG) and from B-splines and linear transformations in the form of a quadratic function, outperformed overall other models on the average of 5 independent training runs. We expect that FC-KAN can leverage function combinations to design future KANs. Our repository is publicly available at: https://github.com/hoangthangta/FC_KAN.

Figures (7)

• Figure 1: Left: The structure of KAN(2,3,1). Right: The simulation of how to calculate $\phi_{1,1,1}$ by control points and B-splines. $G$ and $k$ is the grid size and the spline order, the number of B-splines equals $G + k = 3 + 3 = 6$.
• Figure 2: The plots of the functions when fitted to pass through the 4 chosen points.
• Figure 3: The structure of FC-KAN and the three types of combined outputs: element-wise, concatenation, and linearization.
• Figure 4: Various data combinations are performed using element-wise operations (additions $+$ and multiplications $\odot$) over two given outputs. The outputs always have the same data dimensions as the inputs.
• Figure 5: The logarithmic values of training losses for the models over 25 epochs on MNIST and 35 epochs on Fashion-MNIST. A quadratic function is used to combine B-splines and DoG at the output of FC-KAN.