Enhancing Federated Learning with Kolmogorov-Arnold Networks: A Comparative Study Across Diverse Aggregation Strategies

TL;DR

This study investigates the performance of Kolmogorov-Arnold Networks ($KAN$) within federated learning (FL) and compares them to traditional Multilayer Perceptrons (MLP) across four diverse tasks. It implements a Federated Kolmogorov-Arnold Network framework ($F$-KAN) and evaluates multiple global aggregation strategies, highlighting that $KAN$ consistently achieves higher accuracy, stability, and faster convergence, particularly with trimmed mean and FedProx settings. The experiments demonstrate that $KAN$ is robust to increasing client counts and non-IID distributions, offering a more reliable and scalable alternative for privacy-preserving decentralized learning. The findings suggest practical benefits for deploying $KAN$ in distributed environments and inform aggregation strategy choices for optimized performance.

Abstract

Multilayer Perceptron (MLP), as a simple yet powerful model, continues to be widely used in classification and regression tasks. However, traditional MLPs often struggle to efficiently capture nonlinear relationships in load data when dealing with complex datasets. Kolmogorov-Arnold Networks (KAN), inspired by the Kolmogorov-Arnold representation theorem, have shown promising capabilities in modeling complex nonlinear relationships. In this study, we explore the performance of KANs within federated learning (FL) frameworks and compare them to traditional Multilayer Perceptrons. Our experiments, conducted across four diverse datasets demonstrate that KANs consistently outperform MLPs in terms of accuracy, stability, and convergence efficiency. KANs exhibit remarkable robustness under varying client numbers and non-IID data distributions, maintaining superior performance even as client heterogeneity increases. Notably, KANs require fewer communication rounds to converge compared to MLPs, highlighting their efficiency in FL scenarios. Additionally, we evaluate multiple parameter aggregation strategies, with trimmed mean and FedProx emerging as the most effective for optimizing KAN performance. These findings establish KANs as a robust and scalable alternative to MLPs for federated learning tasks, paving the way for their application in decentralized and privacy-preserving environments.

Figures (4)

• Figure 1: (a) The proposed framework for Federated Kolmogorov-Arnold Networks (F-KAN), demonstrating the interaction between clients, aggregation strategies, and the global model. The dotted and solid arrows represent bidirectional communication in the federated learning process. (b) Icons representing the four datasets used in this study: Gender Classification, Airline Passenger Satisfaction, Cardiovascular Disease Diagnosis, and Weather Type Classification. (c) Visualization of the two models compared in this research: a standard Multilayer Perceptron on the left and the Kolmogorov-Arnold Network on the right (adopted from liu2024kan), highlighting their structural differences.
• Figure 2: (Left) Dataset Airline MLP with 5 clients. (Right) Dataset Airline KAN with 5 clients.
• Figure 3: Impact of Varying Client Numbers on Model Performance
• Figure 4: Performance Comparison of KAN and MLP Across Different Datasets. The left figure presents the average performance of KAN and MLP across all aggregation strategies. The right figure highlights the best performance achieved by each model under the most optimal aggregation strategy