Understanding Server-Assisted Federated Learning in the Presence of Incomplete Client Participation

TL;DR

This work tackles the challenge of incomplete client participation in Federated Learning (FL) by proving a worst-case impossibility for conventional FL to be PAC-learnable under such participation gaps. It introduces Server-Assisted Federated Learning (SA-FL), which augments the server with a small auxiliary dataset to restore PAC-learnability under mild distributional assumptions, and presents a generalization bound that links performance to the total server-plus-client data $n_T+n_S$. To translate theory into practice, the authors propose SAFARI, a two-mode update algorithm that combines client-side FedAvg-like updates with server-side SGD on the auxiliary data, and establish convergence guarantees: $\mathcal{O}(1/\sqrt{R})$ to stationary points in the non-convex case (with potential $\mathcal{O}(1/\sqrt{mKR})$ speedups) and $\mathcal{O}(1/R)$ in the strongly convex setting. Empirical results on MNIST, CIFAR-10, and Fashion-MNIST corroborate the theory, showing substantial accuracy gains and faster convergence under incomplete participation when modest server data are used. Overall, the paper provides both theoretical justification and a practical, convergent approach for robust FL in the presence of system-induced participation gaps.

Abstract

Existing works in federated learning (FL) often assume an ideal system with either full client or uniformly distributed client participation. However, in practice, it has been observed that some clients may never participate in FL training (aka incomplete client participation) due to a myriad of system heterogeneity factors. A popular approach to mitigate impacts of incomplete client participation is the server-assisted federated learning (SA-FL) framework, where the server is equipped with an auxiliary dataset. However, despite SA-FL has been empirically shown to be effective in addressing the incomplete client participation problem, there remains a lack of theoretical understanding for SA-FL. Meanwhile, the ramifications of incomplete client participation in conventional FL are also poorly understood. These theoretical gaps motivate us to rigorously investigate SA-FL. Toward this end, we first show that conventional FL is {\em not} PAC-learnable under incomplete client participation in the worst case. Then, we show that the PAC-learnability of FL with incomplete client participation can indeed be revived by SA-FL, which theoretically justifies the use of SA-FL for the first time. Lastly, to provide practical guidance for SA-FL training under {\em incomplete client participation}, we propose the $\mathsf{SAFARI}$ (server-assisted federated averaging) algorithm that enjoys the same linear convergence speedup guarantees as classic FL with ideal client participation assumptions, offering the first SA-FL algorithm with convergence guarantee. Extensive experiments on different datasets show $\mathsf{SAFARI}$ significantly improves the performance under incomplete client participation.

Figures (2)

• Figure 1: Diagram of distribution supports for domain adaptation and federated learning.
• Figure 2: Comparison of test accuracy on CIFAR-10 ($s=4$, $p=1$, $q=0.4$).

Theorems & Definitions (31)

• Definition 1: Generalization and Empirical Errors
• Definition 2: Optimal Hypothesis
• Definition 3: Excess Error
• Definition 4
• theorem 1: Impossibility Theorem
• proof : Proof Sketch
• theorem 2: Generalization Error Bound for SA-FL
• theorem 3: Conditions of SA-FL Being No Worse Than Centralized Learning
• Remark 1
• Remark 2