A Theoretical Analysis of Efficiency Constrained Utility-Privacy Bi-Objective Optimization in Federated Learning
Hanlin Gu, Xinyuan Zhao, Gongxi Zhu, Yuxing Han, Yan Kang, Lixin Fan, Qiang Yang
TL;DR
This work tackles the DPFL privacy-utility-efficiency trade-off by formulating a constrained bi-objective optimization over the trio $(\sigma, T, q)$ to balance privacy leakage $\epsilon_p$, utility loss $\epsilon_u$, and training efficiency $\epsilon_e$. It derives an analytical Pareto front, anchored by the key relation $k\sigma^2 T = qK$, and shows how to reduce the problem to tractable forms under various constraints. Theoretical results are complemented by extensive experiments on MNIST (LR/LeNet) and CIFAR-10 (ResNet-18), with strong alignment between theory and empirical Pareto fronts, enabling low-cost parameter design for DPFL. The findings offer a principled pathway to jointly optimize privacy, utility, and training time in real deployments, reducing search costs and informing practical DPFL configurations.
Abstract
Federated learning (FL) enables multiple clients to collaboratively learn a shared model without sharing their individual data. Concerns about utility, privacy, and training efficiency in FL have garnered significant research attention. Differential privacy has emerged as a prevalent technique in FL, safeguarding the privacy of individual user data while impacting utility and training efficiency. Within Differential Privacy Federated Learning (DPFL), previous studies have primarily focused on the utility-privacy trade-off, neglecting training efficiency, which is crucial for timely completion. Moreover, differential privacy achieves privacy by introducing controlled randomness (noise) on selected clients in each communication round. Previous work has mainly examined the impact of noise level ($σ$) and communication rounds ($T$) on the privacy-utility dynamic, overlooking other influential factors like the sample ratio ($q$, the proportion of selected clients). This paper systematically formulates an efficiency-constrained utility-privacy bi-objective optimization problem in DPFL, focusing on $σ$, $T$, and $q$. We provide a comprehensive theoretical analysis, yielding analytical solutions for the Pareto front. Extensive empirical experiments verify the validity and efficacy of our analysis, offering valuable guidance for low-cost parameter design in DPFL.