Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

"What do you want from theory alone?" Experimenting with Tight Auditing of Differentially Private Synthetic Data Generation

Meenatchi Sundaram Muthu Selva Annamalai, Georgi Ganev, Emiliano De Cristofaro

TL;DR

This work presents a comprehensive audit framework for six state-of-the-art DP-SDG implementations, using multiple MIAs and threat models to quantify empirical privacy leakage. It demonstrates that black-box MIAs often miss leaks and metadata-based violations, while white-box and active white-box attacks—especially with worst-case datasets—produce markedly tighter estimates and reveal DP violations (including a new one in DPWGAN). The study underscores the necessity of implementation-specific worst-case inputs and stronger adversaries to reliably bound actual privacy loss in practice, and it argues for automated auditing as a scalable, CI-friendly tool to verify DP guarantees in real-world releases.

Abstract

Differentially private synthetic data generation (DP-SDG) algorithms are used to release datasets that are structurally and statistically similar to sensitive data while providing formal bounds on the information they leak. However, bugs in algorithms and implementations may cause the actual information leakage to be higher. This prompts the need to verify whether the theoretical guarantees of state-of-the-art DP-SDG implementations also hold in practice. We do so via a rigorous auditing process: we compute the information leakage via an adversary playing a distinguishing game and running membership inference attacks (MIAs). If the leakage observed empirically is higher than the theoretical bounds, we identify a DP violation; if it is non-negligibly lower, the audit is loose. We audit six DP-SDG implementations using different datasets and threat models and find that black-box MIAs commonly used against DP-SDGs are severely limited in power, yielding remarkably loose empirical privacy estimates. We then consider MIAs in stronger threat models, i.e., passive and active white-box, using both existing and newly proposed attacks. Overall, we find that, currently, we do not only need white-box MIAs but also worst-case datasets to tightly estimate the privacy leakage from DP-SDGs. Finally, we show that our automated auditing procedure finds both known DP violations (in 4 out of the 6 implementations) as well as a new one in the DPWGAN implementation that was successfully submitted to the NIST DP Synthetic Data Challenge. The source code needed to reproduce our experiments is available from https://github.com/spalabucr/synth-audit.

"What do you want from theory alone?" Experimenting with Tight Auditing of Differentially Private Synthetic Data Generation

TL;DR

This work presents a comprehensive audit framework for six state-of-the-art DP-SDG implementations, using multiple MIAs and threat models to quantify empirical privacy leakage. It demonstrates that black-box MIAs often miss leaks and metadata-based violations, while white-box and active white-box attacks—especially with worst-case datasets—produce markedly tighter estimates and reveal DP violations (including a new one in DPWGAN). The study underscores the necessity of implementation-specific worst-case inputs and stronger adversaries to reliably bound actual privacy loss in practice, and it argues for automated auditing as a scalable, CI-friendly tool to verify DP guarantees in real-world releases.

Abstract

Differentially private synthetic data generation (DP-SDG) algorithms are used to release datasets that are structurally and statistically similar to sensitive data while providing formal bounds on the information they leak. However, bugs in algorithms and implementations may cause the actual information leakage to be higher. This prompts the need to verify whether the theoretical guarantees of state-of-the-art DP-SDG implementations also hold in practice. We do so via a rigorous auditing process: we compute the information leakage via an adversary playing a distinguishing game and running membership inference attacks (MIAs). If the leakage observed empirically is higher than the theoretical bounds, we identify a DP violation; if it is non-negligibly lower, the audit is loose. We audit six DP-SDG implementations using different datasets and threat models and find that black-box MIAs commonly used against DP-SDGs are severely limited in power, yielding remarkably loose empirical privacy estimates. We then consider MIAs in stronger threat models, i.e., passive and active white-box, using both existing and newly proposed attacks. Overall, we find that, currently, we do not only need white-box MIAs but also worst-case datasets to tightly estimate the privacy leakage from DP-SDGs. Finally, we show that our automated auditing procedure finds both known DP violations (in 4 out of the 6 implementations) as well as a new one in the DPWGAN implementation that was successfully submitted to the NIST DP Synthetic Data Challenge. The source code needed to reproduce our experiments is available from https://github.com/spalabucr/synth-audit.
Paper Structure (39 sections, 2 theorems, 6 equations, 13 figures, 2 tables, 1 algorithm)

This paper contains 39 sections, 2 theorems, 6 equations, 13 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Differential Privacy (DP)
  4. Auditing DP
  5. Membership Inference Attacks (MIAs)
  6. Synthetic Data Generation (SDG)
  7. Auditing DP-SDG Algorithms
  8. Overview
  9. Threat Models
  10. DP Distinguishing Game
  11. Worst-Case Target Record
  12. Worst-Case Neighboring Datasets
  13. Evaluation Framework
  14. Datasets
  15. DP-SDG Algorithms
  16. ...and 24 more sections

Key Result

Theorem 1

Let $\mathcal{M}: \mathcal{D} \rightarrow \mathcal{R}$ be an ($\varepsilon, \delta$)-DP mechanism and $f: \; \mathcal{R} \rightarrow \mathcal{R}'$. Then $f \circ \mathcal{M}: \mathcal{D} \rightarrow \mathcal{R}'$ also satisfies $(\epsilon, \delta)$-DP.

Figures (13)

  • Figure 1: Distinguishability Game between Adversary and Challenger for add/remove DP, given a raw dataset ($\mathcal{D}$), the number of records in the original dataset ($n$), the number of records in the synthetic dataset ($m$), the generative model fitting function ($\text{GM}$), and a decision threshold $\tau$.
  • Figure 2: Choosing the worst-case target record to audit.
  • Figure 3: Black-box auditing, Querybased and DCR attacks.
  • Figure 4: Distribution of Querybased/DCR attack scores against PrivBayes (Hazy) trained on $D$ vs $D'$ at $\varepsilon = 1.0$.
  • Figure 5: Black-box auditing at $\varepsilon = 4.0$ with different worst-case datasets using the Querybased attack.
  • ...and 8 more figures

Theorems & Definitions (3)

  • Definition 1: Differential Privacy (DP) dwork2006calibrating
  • Theorem 1: Post-Processing
  • Theorem 2: $\mu$-GDP to ($\varepsilon, \delta$)-DP conversion dong2019gaussian