The More is not the Merrier: Investigating the Effect of Client Size on Federated Learning

TL;DR

This study examines how the number of participating clients $K$ affects FedAvg accuracy in Federated Learning, particularly in Mobile Edge Computing and cross-silo contexts. It identifies a clear accuracy deterioration as $K$ increases and introduces Knowledgeable Client Insertion (KCI), which injects $m$ data-rich artificial clients with data proportion controlled by $oldsymbol{\\lambda}$ to accelerate learning under standard FedAvg. CIFAR-10 experiments demonstrate substantial gains, up to about 45% improvement for large $K$, validating KCI as a practical approach to scale FL while preserving privacy. The work highlights a path to maintain performance in large-scale FL and suggests future directions for privacy analysis and compatibility with other aggregation schemes in non-IID settings.

Abstract

Federated Learning (FL) has been introduced as a way to keep data local to clients while training a shared machine learning model, as clients train on their local data and send trained models to a central aggregator. It is expected that FL will have a huge implication on Mobile Edge Computing, the Internet of Things, and Cross-Silo FL. In this paper, we focus on the widely used FedAvg algorithm to explore the effect of the number of clients in FL. We find a significant deterioration of learning accuracy for FedAvg as the number of clients increases. To address this issue for a general application, we propose a method called Knowledgeable Client Insertion (KCI) that introduces a very small number of knowledgeable clients to the MEC setting. These knowledgeable clients are expected to have accumulated a large set of data samples to help with training. With the help of KCI, the learning accuracy of FL increases much faster even with a normal FedAvg aggregation technique. We expect this approach to be able to provide great privacy protection for clients against security attacks such as model inversion attacks. Our code is available at https://github.com/Eleanor-W/KCI_for_FL.

Figures (7)

• Figure 1: An Illustration of Federated Learning Framework.
• Figure 2: Accuracy for different $K$ of FedAvg
• Figure 3: Accuracy comparison between FedAvg and KCI differing $K$ ($m=1$ and $\lambda=1$).
• Figure 4: Accuracy for different $\lambda$ for KCI ($m=1$ and $K=10$).
• Figure 5: Accuracy for different $\lambda$ and $K$ ($m=1$).