Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

JSAM: Privacy Straggler-Resilient Joint Client Selection and Incentive Mechanism Design in Differentially Private Federated Learning

Ruichen Xu, Ying-Jun Angela Zhang, Jianwei Huang

TL;DR

It is proved that servers should preferentially select privacy-tolerant clients while excluding high-sensitivity participants, and the counter-intuitive insight that clients with minimal privacy sensitivity may incur the highest cumulative costs due to frequent participation is uncovered.

Abstract

Differentially private federated learning faces a fundamental tension: privacy protection mechanisms that safeguard client data simultaneously create quantifiable privacy costs that discourage participation, undermining the collaborative training process. Existing incentive mechanisms rely on unbiased client selection, forcing servers to compensate even the most privacy-sensitive clients ("privacy stragglers"), leading to systemic inefficiency and suboptimal resource allocation. We introduce JSAM (Joint client Selection and privacy compensAtion Mechanism), a Bayesian-optimal framework that simultaneously optimizes client selection probabilities and privacy compensation to maximize training effectiveness under budget constraints. Our approach transforms a complex 2N-dimensional optimization problem into an efficient three-dimensional formulation through novel theoretical characterization of optimal selection strategies. We prove that servers should preferentially select privacy-tolerant clients while excluding high-sensitivity participants, and uncover the counter-intuitive insight that clients with minimal privacy sensitivity may incur the highest cumulative costs due to frequent participation. Extensive evaluations on MNIST and CIFAR-10 demonstrate that JSAM achieves up to 15% improvement in test accuracy compared to existing unbiased selection mechanisms while maintaining cost efficiency across varying data heterogeneity levels.

JSAM: Privacy Straggler-Resilient Joint Client Selection and Incentive Mechanism Design in Differentially Private Federated Learning

TL;DR

It is proved that servers should preferentially select privacy-tolerant clients while excluding high-sensitivity participants, and the counter-intuitive insight that clients with minimal privacy sensitivity may incur the highest cumulative costs due to frequent participation is uncovered.

Abstract

Differentially private federated learning faces a fundamental tension: privacy protection mechanisms that safeguard client data simultaneously create quantifiable privacy costs that discourage participation, undermining the collaborative training process. Existing incentive mechanisms rely on unbiased client selection, forcing servers to compensate even the most privacy-sensitive clients ("privacy stragglers"), leading to systemic inefficiency and suboptimal resource allocation. We introduce JSAM (Joint client Selection and privacy compensAtion Mechanism), a Bayesian-optimal framework that simultaneously optimizes client selection probabilities and privacy compensation to maximize training effectiveness under budget constraints. Our approach transforms a complex 2N-dimensional optimization problem into an efficient three-dimensional formulation through novel theoretical characterization of optimal selection strategies. We prove that servers should preferentially select privacy-tolerant clients while excluding high-sensitivity participants, and uncover the counter-intuitive insight that clients with minimal privacy sensitivity may incur the highest cumulative costs due to frequent participation. Extensive evaluations on MNIST and CIFAR-10 demonstrate that JSAM achieves up to 15% improvement in test accuracy compared to existing unbiased selection mechanisms while maintaining cost efficiency across varying data heterogeneity levels.
Paper Structure (46 sections, 12 theorems, 67 equations, 8 figures, 3 tables, 1 algorithm)

This paper contains 46 sections, 12 theorems, 67 equations, 8 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Incentive Mechanism Design for Federated Learning
  5. Differentially Private Federated Learning
  6. Incentive Mechanism Design in Differentially Private Federated Learning
  7. Preliminary
  8. Federated Learning
  9. Differential Privacy
  10. Differentially Private Federated Learning
  11. System Model
  12. Client Payoff
  13. Server Payoff
  14. Joint Client Selection and Privacy Compensation Mechanism Design
  15. JSAM
  16. ...and 31 more sections

Key Result

Lemma 1

There exist constants $c_1$ and $c_2$ so that for any $\epsilon < c_1T_0$, adding $\mathcal{N}(0,\sigma^2)$ noise on the output in each iteration is $(\epsilon, \delta)$-differentially private for any $\delta > 0$ if where $T_0$ is the number of iterations.

Figures (8)

  • Figure 1: Differentially private federated learning employs noisy gradient updates to safeguard client privacy.
  • Figure 2: The "privacy straggler" effect. A single client with very low privacy budget ($\epsilon_k$) forces the server to add excessive noise, degrade the model utility.
  • Figure 3: Workflow illustration of JSAM.
  • Figure 4: Flow chart of problem reformulations.
  • Figure 5: An illustration of the structure of the optimal client selection probabilities with ten clients $(v_1 < \cdots < v_{10})$.
  • ...and 3 more figures

Theorems & Definitions (23)

  • Definition 1: Differential privacy
  • Lemma 1: Noise varianceabadi2016deep
  • Theorem 1: Convergence analysisxu
  • Definition 2: Incentive compatibility - IC
  • Definition 3: Individual rationality - IR
  • Lemma 2
  • Definition 4: Virtual cost
  • Theorem 2
  • Proposition 1
  • Theorem 3
  • ...and 13 more