Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ATTAXONOMY: Unpacking Differential Privacy Guarantees Against Practical Adversaries

Rachel Cummings, Shlomi Hod, Jayshree Sarathy, Marika Swanberg

TL;DR

This work tackles the interpretability gap of differential privacy by introducing a detailed taxonomy of privacy attacks that accounts for attacker knowledge, goals, and evaluation metrics. It demonstrates how threat modeling can contextualize DP parameters, using Israel's birth-data release as a concrete case study and introducing DistReRo (distributional reconstruction robustness) to model average-case protections. The authors connect reconstruction notions to DP guarantees, showing that DP implies both ReRo and DistReRo under certain conditions, while DistReRo remains strictly weaker than ReRo. Overall, the paper contributes a practical framework for assessing real-world privacy risks under DP, guiding parameter selection and risk communication in deployment contexts. This work advances the DP discourse by aligning theoretical guarantees with realistic adversaries and risk-management workflows, potentially improving policy decisions around privacy budgets in practice.

Abstract

Differential Privacy (DP) is a mathematical framework that is increasingly deployed to mitigate privacy risks associated with machine learning and statistical analyses. Despite the growing adoption of DP, its technical privacy parameters do not lend themselves to an intelligible description of the real-world privacy risks associated with that deployment: the guarantee that most naturally follows from the DP definition is protection against membership inference by an adversary who knows all but one data record and has unlimited auxiliary knowledge. In many settings, this adversary is far too strong to inform how to set real-world privacy parameters. One approach for contextualizing privacy parameters is via defining and measuring the success of technical attacks, but doing so requires a systematic categorization of the relevant attack space. In this work, we offer a detailed taxonomy of attacks, showing the various dimensions of attacks and highlighting that many real-world settings have been understudied. Our taxonomy provides a roadmap for analyzing real-world deployments and developing theoretical bounds for more informative privacy attacks. We operationalize our taxonomy by using it to analyze a real-world case study, the Israeli Ministry of Health's recent release of a birth dataset using DP, showing how the taxonomy enables fine-grained threat modeling and provides insight towards making informed privacy parameter choices. Finally, we leverage the taxonomy towards defining a more realistic attack than previously considered in the literature, namely a distributional reconstruction attack: we generalize Balle et al.'s notion of reconstruction robustness to a less-informed adversary with distributional uncertainty, and extend the worst-case guarantees of DP to this average-case setting.

ATTAXONOMY: Unpacking Differential Privacy Guarantees Against Practical Adversaries

TL;DR

This work tackles the interpretability gap of differential privacy by introducing a detailed taxonomy of privacy attacks that accounts for attacker knowledge, goals, and evaluation metrics. It demonstrates how threat modeling can contextualize DP parameters, using Israel's birth-data release as a concrete case study and introducing DistReRo (distributional reconstruction robustness) to model average-case protections. The authors connect reconstruction notions to DP guarantees, showing that DP implies both ReRo and DistReRo under certain conditions, while DistReRo remains strictly weaker than ReRo. Overall, the paper contributes a practical framework for assessing real-world privacy risks under DP, guiding parameter selection and risk communication in deployment contexts. This work advances the DP discourse by aligning theoretical guarantees with realistic adversaries and risk-management workflows, potentially improving policy decisions around privacy budgets in practice.

Abstract

Differential Privacy (DP) is a mathematical framework that is increasingly deployed to mitigate privacy risks associated with machine learning and statistical analyses. Despite the growing adoption of DP, its technical privacy parameters do not lend themselves to an intelligible description of the real-world privacy risks associated with that deployment: the guarantee that most naturally follows from the DP definition is protection against membership inference by an adversary who knows all but one data record and has unlimited auxiliary knowledge. In many settings, this adversary is far too strong to inform how to set real-world privacy parameters. One approach for contextualizing privacy parameters is via defining and measuring the success of technical attacks, but doing so requires a systematic categorization of the relevant attack space. In this work, we offer a detailed taxonomy of attacks, showing the various dimensions of attacks and highlighting that many real-world settings have been understudied. Our taxonomy provides a roadmap for analyzing real-world deployments and developing theoretical bounds for more informative privacy attacks. We operationalize our taxonomy by using it to analyze a real-world case study, the Israeli Ministry of Health's recent release of a birth dataset using DP, showing how the taxonomy enables fine-grained threat modeling and provides insight towards making informed privacy parameter choices. Finally, we leverage the taxonomy towards defining a more realistic attack than previously considered in the literature, namely a distributional reconstruction attack: we generalize Balle et al.'s notion of reconstruction robustness to a less-informed adversary with distributional uncertainty, and extend the worst-case guarantees of DP to this average-case setting.
Paper Structure (27 sections, 6 theorems, 13 equations, 2 figures, 1 table, 2 algorithms)

This paper contains 27 sections, 6 theorems, 13 equations, 2 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Taxonomies of privacy attacks.
  4. Recent work on reconstruction attacks.
  5. Our Contributions
  6. Proposing a taxonomy to characterize privacy attacks.
  7. Using our taxonomy to analyze Israel's Birth Dataset Release.
  8. Defining and characterizing reconstruction success against distribution-only adversaries.
  9. Characterizing the space of attacks: Our Taxonomy
  10. Crafter
  11. Attacker
  12. Evaluator
  13. Case Study: Taxonomy in Action
  14. Microdata Release
  15. Threat Modeling
  16. ...and 12 more sections

Key Result

Theorem 4.3

Fix any prior $\mathcal{D}\xspace$ over $\mathcal{X}$, $\eta>0$, loss function $\ell:\mathcal{X}\xspace \times \mathcal{X}\xspace \to \mathbb{R}\xspace_{\geq 0}$, and corresponding reconstruction robustness baseline error $\kappa = \kappa_{\mathcal{D}\xspace, \ell}(\eta)$. If $\mathcal{M}\xspace$ is

Figures (2)

  • Figure 1: Visualization attacks with the taxonomy of the privacy attack taxonomy. The dimensions are represented as vertical bars and grouped according to their associated roles. Attacks are depicted as lines that cross all dimensions.
  • Figure 2: Summary of the relationships among different notions of reconstruction robustness for fixed reconstruction loss $\eta$ throughout. Our new definitions are highlighted in yellow, and our new implications among definitions are highlighted in blue. The red dashed arrow indicates that the implication does not hold.

Theorems & Definitions (17)

  • Definition 4.1: Reconstruction Robustness (ReRo) balle2022reconstructing
  • Definition 4.2: ReRo Baseline Error balle2022reconstructing
  • Theorem 4.3: balle2022reconstructing, Theorem 2
  • Definition 4.4: Best Case Distributional Reconstruction Robustness, BCDistReRo
  • Definition 4.5: BCDistReRo Baseline Success
  • Theorem 4.6
  • proof
  • Corollary 4.7
  • proof
  • Definition 4.8: Average Distributional Reconstruction Robustness
  • ...and 7 more