FedClust: Optimizing Federated Learning on Non-IID Data through Weight-Driven Client Clustering

TL;DR

FedClust groups clients into clusters in a one-shot manner using strategically selected partial model weights and dynamically accommodates newcomers in real-time, and outperforms baseline approaches in terms of accuracy and communication costs.

Abstract

Federated learning (FL) is an emerging distributed machine learning paradigm enabling collaborative model training on decentralized devices without exposing their local data. A key challenge in FL is the uneven data distribution across client devices, violating the well-known assumption of independent-and-identically-distributed (IID) training samples in conventional machine learning. Clustered federated learning (CFL) addresses this challenge by grouping clients based on the similarity of their data distributions. However, existing CFL approaches require a large number of communication rounds for stable cluster formation and rely on a predefined number of clusters, thus limiting their flexibility and adaptability. This paper proposes FedClust, a novel CFL approach leveraging correlations between local model weights and client data distributions. FedClust groups clients into clusters in a one-shot manner using strategically selected partial model weights and dynamically accommodates newcomers in real-time. Experimental results demonstrate FedClust outperforms baseline approaches in terms of accuracy and communication costs.

Figures (2)

• Figure 1: Illustration of the distance matrices calculated using different layer weights (CL: convolutional layer, FL: fully connected layer). Lighter color indicates smaller distances, i.e., the two models are more similar.
• Figure 2: An overview of FedClust.