Multi-Objective Optimization for Privacy-Utility Balance in Differentially Private Federated Learning
Kanishka Ranaweera, David Smith, Pubudu N. Pathirana, Ming Ding, Thierry Rakotoarivelo, Aruna Seneviratne
TL;DR
The paper addresses the privacy–utility trade-off in differentially private federated learning by introducing an adaptive clipping mechanism that dynamically adjusts the gradient clipping norm $C$ through a multi-objective optimization framework. It establishes a composite objective $\mathcal{L}=\mathcal{L}_{\text{model}}+\kappa\mathcal{L}_{\text{clipping}}$ and derives a gradient-based update for $C$, along with a convexity/PL-based convergence analysis. The methodology couples sample-level DP with dynamic clipping to reduce unnecessary clipping while maintaining privacy guarantees, and the convergence analysis shows how DP noise and learning rates influence progress. Empirical results on MNIST, Fashion-MNIST, and CIFAR-10 demonstrate that DP-FL with MOO generally outperforms state-of-the-art baselines under various privacy budgets, with notable improvements under stricter privacy, underscoring the practical value of adaptive clipping in privacy-preserving FL.
Abstract
Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, making it a promising approach for privacy-preserving machine learning. However, ensuring differential privacy (DP) in FL presents challenges due to the trade-off between model utility and privacy protection. Clipping gradients before aggregation is a common strategy to limit privacy loss, but selecting an optimal clipping norm is non-trivial, as excessively high values compromise privacy, while overly restrictive clipping degrades model performance. In this work, we propose an adaptive clipping mechanism that dynamically adjusts the clipping norm using a multi-objective optimization framework. By integrating privacy and utility considerations into the optimization objective, our approach balances privacy preservation with model accuracy. We theoretically analyze the convergence properties of our method and demonstrate its effectiveness through extensive experiments on MNIST, Fashion-MNIST, and CIFAR-10 datasets. Our results show that adaptive clipping consistently outperforms fixed-clipping baselines, achieving improved accuracy under the same privacy constraints. This work highlights the potential of dynamic clipping strategies to enhance privacy-utility trade-offs in differentially private federated learning.