Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Joint Selection: Adaptively Incorporating Public Information for Private Synthetic Data

Miguel Fuentes, Brett Mullins, Ryan McKenna, Gerome Miklau, Daniel Sheldon

TL;DR

The mechanism jam-pgm is developed, which expands the adaptive measurements framework to jointly select between measuring public data and private data and is able to outperform both publicly assisted and non publicly assisted synthetic data generation mechanisms even when the public data distribution is biased.

Abstract

Mechanisms for generating differentially private synthetic data based on marginals and graphical models have been successful in a wide range of settings. However, one limitation of these methods is their inability to incorporate public data. Initializing a data generating model by pre-training on public data has shown to improve the quality of synthetic data, but this technique is not applicable when model structure is not determined a priori. We develop the mechanism jam-pgm, which expands the adaptive measurements framework to jointly select between measuring public data and private data. This technique allows for public data to be included in a graphical-model-based mechanism. We show that jam-pgm is able to outperform both publicly assisted and non publicly assisted synthetic data generation mechanisms even when the public data distribution is biased.

Joint Selection: Adaptively Incorporating Public Information for Private Synthetic Data

TL;DR

The mechanism jam-pgm is developed, which expands the adaptive measurements framework to jointly select between measuring public data and private data and is able to outperform both publicly assisted and non publicly assisted synthetic data generation mechanisms even when the public data distribution is biased.

Abstract

Mechanisms for generating differentially private synthetic data based on marginals and graphical models have been successful in a wide range of settings. However, one limitation of these methods is their inability to incorporate public data. Initializing a data generating model by pre-training on public data has shown to improve the quality of synthetic data, but this technique is not applicable when model structure is not determined a priori. We develop the mechanism jam-pgm, which expands the adaptive measurements framework to jointly select between measuring public data and private data. This technique allows for public data to be included in a graphical-model-based mechanism. We show that jam-pgm is able to outperform both publicly assisted and non publicly assisted synthetic data generation mechanisms even when the public data distribution is biased.
Paper Structure (48 sections, 5 theorems, 4 equations, 13 figures, 1 table, 2 algorithms)

This paper contains 48 sections, 5 theorems, 4 equations, 13 figures, 1 table, 2 algorithms.

Table of Contents

  1. INTRODUCTION
  2. BACKGROUND
  3. Differential Privacy
  4. Marginals and Workloads
  5. Private-pgm
  6. JOINT ADAPTIVE MEASUREMENTS
  7. Public Proxy Estimator
  8. Public/Private Measurement
  9. Joint Candidate Set
  10. Expected Improvement Score Function
  11. Frugal Budgeting
  12. Privacy Proof
  13. PRIOR WORK
  14. EXPERIMENTS
  15. Task Specification
  16. ...and 33 more sections

Key Result

Proposition 2.3

If mechanism $\mathcal{M}$ satisfies $\rho$-zCDP, then, it satisfies $(\epsilon, \delta)$-DP for any $\epsilon > 0$ and $\delta = \min_{\alpha > 1} \frac{\exp((\alpha - 1)(\alpha\rho-\epsilon))}{\alpha - 1}\left(1 - \frac{1}{\alpha}\right)^\alpha$.

Figures (13)

  • Figure 1: Average workload error (workload of all 3 way marginals) for $\epsilon$ in {0.03, 0.10, 0.31, 1.00, 3.16, 10.00} and $\delta=1\times10^{-9}$ for the ADULT data set. Private dataset consists of 25% female records while public dataset consists of 100%, 75%, 50%, and 25% female records respectively. The plots are shown with the amount of public data bias decreasing from left to right.
  • Figure 2: Average workload error (workload of all 3-way marginals) for $\epsilon$ in {0.03, 0.10, 0.31, 1.00, 3.16, 10.00} and $\delta=1\times10^{-9}$ for TITANIC data set.
  • Figure 3: Average workload error (workload of all 3-way marginals) for $\epsilon$ in {0.03, 0.10, 0.31, 1.00, 3.16, 10.00} and $\delta=1\times10^{-9}$ for the SALARY , FIRE , and NIST-TAXI datasets.
  • Figure 4: Average workload error for $\textsc{gem}\xspace$ on the private ADULT data set. The colors indicate the setting of the rounds hyperparameter $T$.
  • Figure 5: Average workload error for $\textsc{gem}^\text{Pub}\xspace$ on the ADULT data sets. The colors indicate the setting of the rounds hyperparameter $T$.
  • ...and 8 more figures

Theorems & Definitions (15)

  • Definition 2.1: Differential privacy; DP
  • Definition 2.2: Zero-concentrated differential privacy; zCDP
  • Proposition 2.3: zCDP to DP Conversion; canonne2020discrete
  • Proposition 2.4: zCDP of Gaussian mechanism; bun2016concentrated
  • Proposition 2.5: zCDP of exponential mechanism; cesar2021bounding
  • Proposition 2.6: Fully adaptive composition for zCDP; whitehouse2022fully
  • Definition 2.7
  • Definition 2.8
  • Definition 3.1
  • Definition 3.2
  • ...and 5 more