An Adaptive Clustering Scheme for Client Selections in Communication-Efficient Federated Learning
Yan-Ann Chen, Guan-Lin Chen
TL;DR
To address high communication costs in non-IID federated learning, the paper proposes adaptive clustering that dynamically tunes the cluster count $p$ during training. Clustering is performed with agglomerative hierarchical clustering based on pairwise model similarity, forming $p$ clusters and selecting one representative per cluster for each round. The cluster count is updated by a TCP-inspired rule starting from $p=n$, where a loss-reduction threshold $(L_{i-1}-L_i)/L_{i-1}>w$ triggers $p ightarrow p-d$ with $d$ growing exponentially, otherwise $p ightarrow ext{min}(2p,n)$, complemented by SA-like and experience-based stabilizers to keep $p$ stable. Experiments on 8 non-IID MNIST clients show about a 50% reduction in transmissions while achieving accuracy near 96.2%, demonstrating practical gains in communication efficiency without sacrificing performance.
Abstract
Federated learning is a novel decentralized learning architecture. During the training process, the client and server must continuously upload and receive model parameters, which consumes a lot of network transmission resources. Some methods use clustering to find more representative customers, select only a part of them for training, and at the same time ensure the accuracy of training. However, in federated learning, it is not trivial to know what the number of clusters can bring the best training result. Therefore, we propose to dynamically adjust the number of clusters to find the most ideal grouping results. It may reduce the number of users participating in the training to achieve the effect of reducing communication costs without affecting the model performance. We verify its experimental results on the non-IID handwritten digit recognition dataset and reduce the cost of communication and transmission by almost 50% compared with traditional federated learning without affecting the accuracy of the model.