Enhancing the Performance of Global Model by Improving the Adaptability of Local Models in Federated Learning
Wujun Zhou, Shu Ding, ZeLin Li, Wei Wang
TL;DR
The paper tackles non-IID data in federated learning by introducing local-model adaptability as the average performance across client distributions and proposing FedACD, which combines a constrained local objective with a KL-divergence-based aggregation to favor adaptable local models. It formalizes the local training via a probability matrix P^m and two losses, L1 and L2, to enforce balanced per-class errors and reflect misclassification margins, while using a template-based aggregation score V_m to weight contributions without sacrificing privacy. The approach is augmented with Input Mixup to handle missing classes and a privacy-preserving scalar aggregation signal, and is validated across CIFAR-10/100 and Tiny-ImageNet under various Non-IID settings, outperforming strong baselines in both global accuracy and local adaptability. Overall, FedACD demonstrates that explicitly enhancing local adaptability and intelligent aggregation can substantially boost global performance in heterogeneous FL scenarios, with practical implications for privacy-preserving, distributed learning systems.
Abstract
Federated learning enables the clients to collaboratively train a global model, which is aggregated from local models. Due to the heterogeneous data distributions over clients and data privacy in federated learning, it is difficult to train local models to achieve a well-performed global model. In this paper, we introduce the adaptability of local models, i.e., the average performance of local models on data distributions over clients, and enhance the performance of the global model by improving the adaptability of local models. Since each client does not know the data distributions over other clients, the adaptability of the local model cannot be directly optimized. First, we provide the property of an appropriate local model which has good adaptability on the data distributions over clients. Then, we formalize the property into the local training objective with a constraint and propose a feasible solution to train the local model. Extensive experiments on federated learning benchmarks demonstrate that our method significantly improves the adaptability of local models and achieves a well-performed global model that consistently outperforms the baseline methods.