Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nearly Tight Black-Box Auditing of Differentially Private Machine Learning

Meenatchi Sundaram Muthu Selva Annamalai, Emiliano De Cristofaro

TL;DR

An auditing procedure for the Differentially Private Stochastic Gradient Descent algorithm in the black-box threat model that is substantially tighter than prior work is presented, and can offer valuable insight into how the privacy analysis of DP-SGD could be improved and detect bugs and DP violations in real-world implementations.

Abstract

This paper presents an auditing procedure for the Differentially Private Stochastic Gradient Descent (DP-SGD) algorithm in the black-box threat model that is substantially tighter than prior work. The main intuition is to craft worst-case initial model parameters, as DP-SGD's privacy analysis is agnostic to the choice of the initial model parameters. For models trained on MNIST and CIFAR-10 at theoretical $\varepsilon=10.0$, our auditing procedure yields empirical estimates of $\varepsilon_{emp} = 7.21$ and $6.95$, respectively, on a 1,000-record sample and $\varepsilon_{emp}= 6.48$ and $4.96$ on the full datasets. By contrast, previous audits were only (relatively) tight in stronger white-box models, where the adversary can access the model's inner parameters and insert arbitrary gradients. Overall, our auditing procedure can offer valuable insight into how the privacy analysis of DP-SGD could be improved and detect bugs and DP violations in real-world implementations. The source code needed to reproduce our experiments is available at https://github.com/spalabucr/bb-audit-dpsgd.

Nearly Tight Black-Box Auditing of Differentially Private Machine Learning

TL;DR

An auditing procedure for the Differentially Private Stochastic Gradient Descent algorithm in the black-box threat model that is substantially tighter than prior work is presented, and can offer valuable insight into how the privacy analysis of DP-SGD could be improved and detect bugs and DP violations in real-world implementations.

Abstract

This paper presents an auditing procedure for the Differentially Private Stochastic Gradient Descent (DP-SGD) algorithm in the black-box threat model that is substantially tighter than prior work. The main intuition is to craft worst-case initial model parameters, as DP-SGD's privacy analysis is agnostic to the choice of the initial model parameters. For models trained on MNIST and CIFAR-10 at theoretical , our auditing procedure yields empirical estimates of and , respectively, on a 1,000-record sample and and on the full datasets. By contrast, previous audits were only (relatively) tight in stronger white-box models, where the adversary can access the model's inner parameters and insert arbitrary gradients. Overall, our auditing procedure can offer valuable insight into how the privacy analysis of DP-SGD could be improved and detect bugs and DP violations in real-world implementations. The source code needed to reproduce our experiments is available at https://github.com/spalabucr/bb-audit-dpsgd.
Paper Structure (32 sections, 1 theorem, 3 equations, 7 figures, 3 tables, 2 algorithms)

This paper contains 32 sections, 1 theorem, 3 equations, 7 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Membership Inference Attacks (MIAs).
  4. Auditing DP-SGD.
  5. Background
  6. Differential Privacy (DP)
  7. Differentially Private Machine Learning (DPML)
  8. DP-SGD.
  9. Fine-tuning regime.
  10. DP Auditing
  11. Deriving $\varepsilon_{emp}$.
  12. Privacy region of DP-SGD.
  13. Auditing Procedure
  14. Threat model.
  15. Auditing.
  16. ...and 17 more sections

Key Result

Theorem 1

A mechanism is $\mu$-GDP iff it is $(\varepsilon, \delta(\varepsilon))$-DP for all $\varepsilon \geq 0$, where:

Figures (7)

  • Figure 1: Auditing models with average-case vs worst-case initial parameters at various levels of $\varepsilon$.
  • Figure 2: Comparing the average gradient norms to empirical privacy leakage, $\varepsilon_{emp}$, for models trained on MNIST at $\varepsilon = 10.0$ with increasing number of pre-training epochs.
  • Figure 3: Auditing models trained on varying dataset sizes ($n$) at different values of $\varepsilon$. The full dataset size $|\mathcal{D}|$ is 30,000 for MNIST and 25,000 for CIFAR-10.
  • Figure 4: Auditing models trained with varying gradient clipping norm ($C$) at $\varepsilon = 1.0, 2.0, 4.0, 10.0$.
  • Figure 5: Auditing models when fine-tuning only the last layer using average-case and worst-case initialization of models at various levels of $\varepsilon$.
  • ...and 2 more figures

Theorems & Definitions (2)

  • Definition 1: Differential Privacy (DP) dwork2006calibrating
  • Theorem 1: $\mu$-GDP to $(\varepsilon, \delta)$-DP conversion dong2019gaussian