Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Provable Mutual Benefits from Federated Learning in Privacy-Sensitive Domains

Nikita Tsoy, Anna Mihalkova, Teodora Todorova, Nikola Konstantinov

TL;DR

This paper provides necessary and sufficient conditions for the existence of mutually beneficial protocols in the context of mean estimation and convex stochastic optimization and designs protocols that maximize the total clients' utility, given symmetric privacy preferences.

Abstract

Cross-silo federated learning (FL) allows data owners to train accurate machine learning models by benefiting from each others private datasets. Unfortunately, the model accuracy benefits of collaboration are often undermined by privacy defenses. Therefore, to incentivize client participation in privacy-sensitive domains, a FL protocol should strike a delicate balance between privacy guarantees and end-model accuracy. In this paper, we study the question of when and how a server could design a FL protocol provably beneficial for all participants. First, we provide necessary and sufficient conditions for the existence of mutually beneficial protocols in the context of mean estimation and convex stochastic optimization. We also derive protocols that maximize the total clients' utility, given symmetric privacy preferences. Finally, we design protocols maximizing end-model accuracy and demonstrate their benefits in synthetic experiments.

Provable Mutual Benefits from Federated Learning in Privacy-Sensitive Domains

TL;DR

This paper provides necessary and sufficient conditions for the existence of mutually beneficial protocols in the context of mean estimation and convex stochastic optimization and designs protocols that maximize the total clients' utility, given symmetric privacy preferences.

Abstract

Cross-silo federated learning (FL) allows data owners to train accurate machine learning models by benefiting from each others private datasets. Unfortunately, the model accuracy benefits of collaboration are often undermined by privacy defenses. Therefore, to incentivize client participation in privacy-sensitive domains, a FL protocol should strike a delicate balance between privacy guarantees and end-model accuracy. In this paper, we study the question of when and how a server could design a FL protocol provably beneficial for all participants. First, we provide necessary and sufficient conditions for the existence of mutually beneficial protocols in the context of mean estimation and convex stochastic optimization. We also derive protocols that maximize the total clients' utility, given symmetric privacy preferences. Finally, we design protocols maximizing end-model accuracy and demonstrate their benefits in synthetic experiments.
Paper Structure (67 sections, 24 theorems, 155 equations, 1 figure, 1 algorithm)

This paper contains 67 sections, 24 theorems, 155 equations, 1 figure, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. Contributions
  3. RELATED WORK
  4. Accuracy-Privacy Trade-off in FL
  5. Privacy-Related Incentives in FL
  6. Other Incentives in Federated Learning
  7. GENERAL FRAMEWORK
  8. Federated Learning Protocol
  9. Clients' Evaluations
  10. $\texttt{err}_i\colon P \to [0, \infty)$
  11. $\texttt{leak}_i\colon P \to [0, \infty)$
  12. Clients' Utilities and Rationality
  13. Parametric Utility Form
  14. Participation Constraints
  15. Server's Objectives
  16. ...and 52 more sections

Key Result

Theorem 4.1

$\hat{\mu}_i$ has the following property where $\rho \coloneqq \frac{n}{\sigma^2}$, $\beta_i \coloneqq \frac{1}{\frac{1}{\rho} + \alpha_i^2}$, $\gamma_i \coloneqq \sum_{k \neq i} \beta_k$.

Figures (1)

  • Figure 1: Personalized vs symmetric protocols. Error bars depict the standard deviation of the average effectiveness.

Theorems & Definitions (33)

  • Definition 3.1
  • Definition 3.2
  • Theorem 4.1: Proof in \ref{['sec:dp-mean-predictor-proof']}
  • Theorem 4.2: Proof in \ref{['sec:dp-mean-existence-proof']}
  • Theorem 4.3: Proof in \ref{['sec:dp-sgd-final-acc-proof']}
  • Theorem 4.4: Proof in \ref{['sec:dp-sgd-privacy-proof']}
  • Theorem 4.5: Proof in \ref{['sec:dp-sgd-existence-proof']}
  • Corollary 4.6: Proof in \ref{['sec:dp-sgd-existence-simple-proof']}
  • Theorem 4.7: Proof in \ref{['sec:b-mean-accuracy-proof']}
  • Theorem 4.8: Proof in \ref{['sec:b-mean-privacy-proof']}
  • ...and 23 more