Optimal Strategies for Federated Learning Maintaining Client Privacy
Uday Bhaskar, Varul Srivastava, Avyukta Manjunatha Vummintala, Naresh Manwani, Sujit Gujar
TL;DR
This work analyzes privacy-preserving federated learning using DP-SGD with FedAvg, focusing on the trade-off between model utility and privacy under a fixed privacy budget $(\epsilon,\delta)$. The authors prove that, for a given total training budget $T$, scheduling one local epoch per global round ($E=1$) yields optimal performance and that increasing the number of participating clients $k$ improves utility, pushing the private model toward the non-private baseline. They also quantify how the number of local epochs and the number of clients impact utility and validate the theory with experiments on MNIST, Fashion-MNIST, and CIFAR-10 across multiple architectures and DP budgets. The results offer practical guidance for deploying privacy-preserving FL, suggesting frequent aggregation and larger client pools to maximize performance under differential privacy constraints.
Abstract
Federated Learning (FL) emerged as a learning method to enable the server to train models over data distributed among various clients. These clients are protective about their data being leaked to the server, any other client, or an external adversary, and hence, locally train the model and share it with the server rather than sharing the data. The introduction of sophisticated inferencing attacks enabled the leakage of information about data through access to model parameters. To tackle this challenge, privacy-preserving federated learning aims to achieve differential privacy through learning algorithms like DP-SGD. However, such methods involve adding noise to the model, data, or gradients, reducing the model's performance. This work provides a theoretical analysis of the tradeoff between model performance and communication complexity of the FL system. We formally prove that training for one local epoch per global round of training gives optimal performance while preserving the same privacy budget. We also investigate the change of utility (tied to privacy) of FL models with a change in the number of clients and observe that when clients are training using DP-SGD and argue that for the same privacy budget, the utility improved with increased clients. We validate our findings through experiments on real-world datasets. The results from this paper aim to improve the performance of privacy-preserving federated learning systems.