Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Secure Aggregation is Not Private Against Membership Inference Attacks

Khac-Hoang Ngo, Johan Östman, Giuseppe Durisi, Alexandre Graell i Amat

TL;DR

The numerical results unveil that, contrary to prevailing claims, SecAgg offers weak privacy against membership inference attacks even in a single training round, and underscore the imperative for additional privacy-enhancing mechanisms, such as noise injection, in federated learning.

Abstract

Secure aggregation (SecAgg) is a commonly-used privacy-enhancing mechanism in federated learning, affording the server access only to the aggregate of model updates while safeguarding the confidentiality of individual updates. Despite widespread claims regarding SecAgg's privacy-preserving capabilities, a formal analysis of its privacy is lacking, making such presumptions unjustified. In this paper, we delve into the privacy implications of SecAgg by treating it as a local differential privacy (LDP) mechanism for each local update. We design a simple attack wherein an adversarial server seeks to discern which update vector a client submitted, out of two possible ones, in a single training round of federated learning under SecAgg. By conducting privacy auditing, we assess the success probability of this attack and quantify the LDP guarantees provided by SecAgg. Our numerical results unveil that, contrary to prevailing claims, SecAgg offers weak privacy against membership inference attacks even in a single training round. Indeed, it is difficult to hide a local update by adding other independent local updates when the updates are of high dimension. Our findings underscore the imperative for additional privacy-enhancing mechanisms, such as noise injection, in federated learning.

Secure Aggregation is Not Private Against Membership Inference Attacks

TL;DR

The numerical results unveil that, contrary to prevailing claims, SecAgg offers weak privacy against membership inference attacks even in a single training round, and underscore the imperative for additional privacy-enhancing mechanisms, such as noise injection, in federated learning.

Abstract

Secure aggregation (SecAgg) is a commonly-used privacy-enhancing mechanism in federated learning, affording the server access only to the aggregate of model updates while safeguarding the confidentiality of individual updates. Despite widespread claims regarding SecAgg's privacy-preserving capabilities, a formal analysis of its privacy is lacking, making such presumptions unjustified. In this paper, we delve into the privacy implications of SecAgg by treating it as a local differential privacy (LDP) mechanism for each local update. We design a simple attack wherein an adversarial server seeks to discern which update vector a client submitted, out of two possible ones, in a single training round of federated learning under SecAgg. By conducting privacy auditing, we assess the success probability of this attack and quantify the LDP guarantees provided by SecAgg. Our numerical results unveil that, contrary to prevailing claims, SecAgg offers weak privacy against membership inference attacks even in a single training round. Indeed, it is difficult to hide a local update by adding other independent local updates when the updates are of high dimension. Our findings underscore the imperative for additional privacy-enhancing mechanisms, such as noise injection, in federated learning.
Paper Structure (30 sections, 8 theorems, 20 equations, 4 figures)

This paper contains 30 sections, 8 theorems, 20 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Main contributions.
  3. Related Work
  4. Secure aggregation.
  5. Differential privacy.
  6. Privacy attacks in with .
  7. Preliminaries
  8. Privacy Analysis of Secure Aggregation
  9. Threat Model
  10. Server.
  11. Clients.
  12. Privacy threat.
  13. as a Noiseless Mechanism
  14. Asymptotic Privacy Guarantee
  15. Upper Bounding $\delta_M(\epsilon)$ via Dominating Pairs of Distributions
  16. ...and 15 more sections

Key Result

theorem thmcountertheorem

Assume that $\{{\mathbf{x}}_i\}_{i=1}^n$ are independent, $\|{\mathbf{x}}_i\|_2 = o(\sqrt{n})$ for $i \in [n]$, and $\frac{1}{n}\sum_{i=1}^n\mathrm{Cov}[{\mathbf{x}}_i] \to {\bm{\Sigma}}$ as $n \to \infty$. Then $\frac{1}{\sqrt{n}}(\!\sum_{i=1}^n \!{\mathbf{x}}_i - \mathbb{E}_{}\left[\sum_{i=1}^n \!

Figures (4)

  • Figure 1: The optimal curve of $M$ in (\ref{['eq:mechanism_client_0']}) where each ${\mathbf{x}}_i$ has independent entries, compared with the Gaussian mechanism $G$ with the same noise covariance matrix.
  • Figure 2: The audited vs. trade-off and curve, averaged over $10$ initial models, for in federated learning on the ADULT dataset with homogeneous data partitioning. Here, $d = 210$.
  • Figure 3: Audited vs. trade-off and curves, averaged over $5$ initial models, for federated learning with on the EMNIST Digits dataset with homogeneous data partitioning. Here, $d = 7850$.
  • Figure 4: Same as Fig. \ref{['fig:EMNIST']} but with heterogeneous data partitioning.

Theorems & Definitions (12)

  • definition thmcounterdefinition: Trade-off function
  • definition thmcounterdefinition: Kas11Duc13
  • definition thmcounterdefinition: $f$-
  • theorem thmcountertheorem: Asymptotic noise distribution
  • proof
  • theorem thmcountertheorem: Correlated Gaussian mechanism
  • proposition thmcounterproposition
  • theorem thmcountertheorem: Dominating pair of distributions
  • corollary thmcountercorollary
  • proposition thmcounterproposition: via the trade-off function
  • ...and 2 more