FedAR: Addressing Client Unavailability in Federated Learning with Local Update Approximation and Rectification
Chutian Jiang, Hansong Zhou, Xiaonan Zhang, Shayok Chakraborty
TL;DR
FedAR tackles the partial participation problem in federated learning by using server-side local update approximation with the latest observed client updates as surrogates and applying an inactivity-aware rectification weight to include unavailable clients without extra client computation. The method updates the global model as w_{t+1} = w_t - (η_t / N_t) sum_i G_i[t] ψ_{i,t}, dynamically counting contributing clients and capping the influence of stale updates to avoid bias. Theoretical results establish convergence for both convex and non-convex smooth losses on non-IID data, with O(1/T) rates in the convex case and O(1/√T) rates in the non-convex case, and experiments on MNIST, CIFAR-10, and SVHN show that FedAR outperforms FedAvg, MIFA, FedVARP, and Scaffold, especially under high client unavailability and large client counts. The approach yields improved training loss and test accuracy, reduces bias against infrequently participating clients, and is scalable to real-world, large-scale FL deployments.
Abstract
Federated learning (FL) enables clients to collaboratively train machine learning models under the coordination of a server in a privacy-preserving manner. One of the main challenges in FL is that the server may not receive local updates from each client in each round due to client resource limitations and intermittent network connectivity. The existence of unavailable clients severely deteriorates the overall FL performance. In this paper, we propose , a novel client update Approximation and Rectification algorithm for FL to address the client unavailability issue. FedAR can get all clients involved in the global model update to achieve a high-quality global model on the server, which also furnishes accurate predictions for each client. To this end, the server uses the latest update from each client as a surrogate for its current update. It then assigns a different weight to each client's surrogate update to derive the global model, in order to guarantee contributions from both available and unavailable clients. Our theoretical analysis proves that FedAR achieves optimal convergence rates on non-IID datasets for both convex and non-convex smooth loss functions. Extensive empirical studies show that FedAR comprehensively outperforms state-of-the-art FL baselines including FedAvg, MIFA, FedVARP and Scaffold in terms of the training loss, test accuracy, and bias mitigation. Moreover, FedAR also depicts impressive performance in the presence of a large number of clients with severe client unavailability.