Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bayes' capacity as a measure for reconstruction attacks in federated learning

Sayan Biswas, Mark Dras, Pedro Faustini, Natasha Fernandes, Annabelle McIver, Catuscia Palamidessi, Parastoo Sadeghi

TL;DR

This work addresses reconstruction attacks in federated learning and critiques using DP-SGD's privacy parameter $\epsilon$ as a standalone metric. It introduces Bayes' capacity, tied to Sibson mutual information of order infinity, as a robust upper bound on leakage for Bayesian attackers, and extends it to continuous mechanisms. The authors derive continuous Bayes' capacity expressions for Gaussian and Von Mises-Fisher DP-SGD mechanisms and validate them with MNIST-like datasets, showing Bayes' capacity aligns better with reconstruction risk than $\epsilon$. The results support Bayes' capacity as a principled metric for mechanism comparison and highlight VMF as a potential alternative to Gaussian noise to improve privacy in FL.

Abstract

Within the machine learning community, reconstruction attacks are a principal attack of concern and have been identified even in federated learning, which was designed with privacy preservation in mind. In federated learning, it has been shown that an adversary with knowledge of the machine learning architecture is able to infer the exact value of a training element given an observation of the weight updates performed during stochastic gradient descent. In response to these threats, the privacy community recommends the use of differential privacy in the stochastic gradient descent algorithm, termed DP-SGD. However, DP has not yet been formally established as an effective countermeasure against reconstruction attacks. In this paper, we formalise the reconstruction threat model using the information-theoretic framework of quantitative information flow. We show that the Bayes' capacity, related to the Sibson mutual information of order infinity, represents a tight upper bound on the leakage of the DP-SGD algorithm to an adversary interested in performing a reconstruction attack. We provide empirical results demonstrating the effectiveness of this measure for comparing mechanisms against reconstruction threats.

Bayes' capacity as a measure for reconstruction attacks in federated learning

TL;DR

This work addresses reconstruction attacks in federated learning and critiques using DP-SGD's privacy parameter as a standalone metric. It introduces Bayes' capacity, tied to Sibson mutual information of order infinity, as a robust upper bound on leakage for Bayesian attackers, and extends it to continuous mechanisms. The authors derive continuous Bayes' capacity expressions for Gaussian and Von Mises-Fisher DP-SGD mechanisms and validate them with MNIST-like datasets, showing Bayes' capacity aligns better with reconstruction risk than . The results support Bayes' capacity as a principled metric for mechanism comparison and highlight VMF as a potential alternative to Gaussian noise to improve privacy in FL.

Abstract

Within the machine learning community, reconstruction attacks are a principal attack of concern and have been identified even in federated learning, which was designed with privacy preservation in mind. In federated learning, it has been shown that an adversary with knowledge of the machine learning architecture is able to infer the exact value of a training element given an observation of the weight updates performed during stochastic gradient descent. In response to these threats, the privacy community recommends the use of differential privacy in the stochastic gradient descent algorithm, termed DP-SGD. However, DP has not yet been formally established as an effective countermeasure against reconstruction attacks. In this paper, we formalise the reconstruction threat model using the information-theoretic framework of quantitative information flow. We show that the Bayes' capacity, related to the Sibson mutual information of order infinity, represents a tight upper bound on the leakage of the DP-SGD algorithm to an adversary interested in performing a reconstruction attack. We provide empirical results demonstrating the effectiveness of this measure for comparing mechanisms against reconstruction threats.
Paper Structure (12 sections, 6 theorems, 14 equations, 2 figures, 2 algorithms)

This paper contains 12 sections, 6 theorems, 14 equations, 2 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. A Formal Model for Reconstruction Attacks
  3. Model for the machine learning system
  4. Model for the attacker
  5. Model for the reconstruction attack
  6. Bayes' Capacity
  7. Bayes' Capacity for Continuous Mechanisms
  8. Comparing Mechanisms
  9. Experimental results
  10. Discussion and Conclusion
  11. The VMF Mechanism
  12. Proofs

Key Result

Lemma 2.1

Let $C$, $D$ be channels such that $C{\cdot}D$ is defined, and $C$ is deterministic. Then $\mathcal{C}_{\mathrm{Bayes}}(C{\cdot}D) = \mathcal{C}_{\mathrm{Bayes}}(D)$.

Figures (2)

  • Figure 1: Reconstruction success in terms of MSE across the datasets using different privacy measures. The left hand figure shows that, for the same epsilon, different mechanisms have different MSE values (a measure of reconstruction success). The right-hand figure shows that, for the same Bayes' capacity, different mechanisms have similar MSE values. Thus, the Bayes' capacity aligns better with reconstruction success when comparing mechanisms.
  • Figure 2: Example of reconstruction success for FMNIST (top) and MNIST (bottom) datasets using $\epsilon = 173$ for the Gaussian and VMF mechanisms.

Theorems & Definitions (13)

  • Remark 1
  • Lemma 2.1
  • Remark 2
  • Definition 3.1: Continuous Bayes' capacity
  • Theorem 3.1: Bayes' Capacity for Gaussian
  • Theorem 4.1: Bayes' Capacity for VMF
  • Definition A.1: VMF mechanism
  • Lemma B.1
  • proof
  • Theorem B.2: Bayes' Capacity for Gaussian
  • ...and 3 more