Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differentially Private Synthetic Data via Foundation Model APIs 1: Images

Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, Harsha Nori, Sergey Yekhanin

TL;DR

This work addresses generating DP synthetic data when only API access to foundation models is available, introducing the training-free Private Evolution (PE) framework. PE iteratively guides API-generated samples toward the private data distribution using a DP Nearest Neighbors Histogram and variation-based mutations, with formal DP guarantees across iterations. Empirically, PE achieves state-of-the-art privacy-utility on CIFAR10 at very low privacy budgets ($\epsilon$ around 0.67) and demonstrates competitive results under large distribution shifts (e.g., Camelyon17) and high-resolution data with Stable Diffusion; it also supports unlimited sample generation for downstream tasks. The approach offers a practical, scalable alternative to training-based DP methods, leveraging public data foundations while preserving data privacy, and opens avenues for DP synthetic data in various modalities and deployment settings.

Abstract

Generating differentially private (DP) synthetic data that closely resembles the original private data is a scalable way to mitigate privacy concerns in the current data-driven world. In contrast to current practices that train customized models for this task, we aim to generate DP Synthetic Data via APIs (DPSDA), where we treat foundation models as blackboxes and only utilize their inference APIs. Such API-based, training-free approaches are easier to deploy as exemplified by the recent surge in the number of API-based apps. These approaches can also leverage the power of large foundation models which are only accessible via their inference APIs. However, this comes with greater challenges due to strictly more restrictive model access and the need to protect privacy from the API provider. In this paper, we present a new framework called Private Evolution (PE) to solve this problem and show its initial promise on synthetic images. Surprisingly, PE can match or even outperform state-of-the-art (SOTA) methods without any model training. For example, on CIFAR10 (with ImageNet as the public data), we achieve FID <= 7.9 with privacy cost ε = 0.67, significantly improving the previous SOTA from ε = 32. We further demonstrate the promise of applying PE on large foundation models such as Stable Diffusion to tackle challenging private datasets with a small number of high-resolution images. The code and data are released at https://github.com/microsoft/DPSDA.

Differentially Private Synthetic Data via Foundation Model APIs 1: Images

TL;DR

This work addresses generating DP synthetic data when only API access to foundation models is available, introducing the training-free Private Evolution (PE) framework. PE iteratively guides API-generated samples toward the private data distribution using a DP Nearest Neighbors Histogram and variation-based mutations, with formal DP guarantees across iterations. Empirically, PE achieves state-of-the-art privacy-utility on CIFAR10 at very low privacy budgets ( around 0.67) and demonstrates competitive results under large distribution shifts (e.g., Camelyon17) and high-resolution data with Stable Diffusion; it also supports unlimited sample generation for downstream tasks. The approach offers a practical, scalable alternative to training-based DP methods, leveraging public data foundations while preserving data privacy, and opens avenues for DP synthetic data in various modalities and deployment settings.

Abstract

Generating differentially private (DP) synthetic data that closely resembles the original private data is a scalable way to mitigate privacy concerns in the current data-driven world. In contrast to current practices that train customized models for this task, we aim to generate DP Synthetic Data via APIs (DPSDA), where we treat foundation models as blackboxes and only utilize their inference APIs. Such API-based, training-free approaches are easier to deploy as exemplified by the recent surge in the number of API-based apps. These approaches can also leverage the power of large foundation models which are only accessible via their inference APIs. However, this comes with greater challenges due to strictly more restrictive model access and the need to protect privacy from the API provider. In this paper, we present a new framework called Private Evolution (PE) to solve this problem and show its initial promise on synthetic images. Surprisingly, PE can match or even outperform state-of-the-art (SOTA) methods without any model training. For example, on CIFAR10 (with ImageNet as the public data), we achieve FID <= 7.9 with privacy cost ε = 0.67, significantly improving the previous SOTA from ε = 32. We further demonstrate the promise of applying PE on large foundation models such as Stable Diffusion to tackle challenging private datasets with a small number of high-resolution images. The code and data are released at https://github.com/microsoft/DPSDA.
Paper Structure (38 sections, 2 theorems, 7 equations, 42 figures, 2 tables, 4 algorithms)

This paper contains 38 sections, 2 theorems, 7 equations, 42 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. DP Synthetic Data via APIs (DPSDA)
  4. Motivation
  5. Problem Formulation
  6. Scope of This Work
  7. Private Evolution (PE)
  8. Conditional Generation
  9. Generating Unlimited Number of Samples
  10. Privacy Analysis
  11. Experiments
  12. Comparisons to State-of-the-Art
  13. Moderate Distribution Shift (ImageNet $\rightarrow$ CIFAR10)
  14. Large Distribution Shift (ImageNet $\rightarrow$ Camelyon17)
  15. Generating Unlimited Number of Samples
  16. ...and 23 more sections

Key Result

Theorem 1

Assume that $\log L \ll d$.If $\log L \gg d\log (D/\eta)$, i.e., if we generate an exponential number of points then by a simple epsilon-net argument we can prove that the algorithm will converge in a single step. With probability $\ge 1-\tau$, the non-private evolution algorithm (alg:main with $\si

Figures (42)

  • Figure 1: We consider the problem of generating DP synthetic data with API access to pre-trained models without any model training. This is in contrast to prior work which assumes full access to pre-trained models and requires training.
  • Figure 2: Private Evolution (PE) framework for DP synthetic data. Left: Intuition of PE. Though private data and pre-trained generative models have very different distributions, the support of the former is likely to be covered by the support of the latter. We gradually shift the distribution of generated data toward private data through PE. Right: Algorithm of PE. We maintain a sample set (population), and iteratively select the most similar ones to the private samples (parents) and mutate them to generate the next population (offspring). The initial population and offspring are generated with foundation model APIs. Parent selection is done in DP using private samples.
  • Figure 3: Generated samples on CIFAR10 with $\left( 0.67,10^{-5} \right)$-DP. Each row corresponds to one class. FID=7.87. See \ref{['app:cifar']} for real and generated images side-by-side.
  • Figure 4: FID heusel2017gans (lower is better) v.s. privacy cost $\epsilon$ on CIFAR10 ($\delta=10^{-5}$). (Un)cond means (un)conditional generation. Ours achieves the best privacy-quality trade-off compared to DP-MEPF harder2022differentially, DP-GAN, DP-Diffusion ghalebikesabi2023differentially.
  • Figure 5: Downstream classification accuracy (higher is better) on CIFAR10 ($\delta=10^{-5}$). The baseline results are taken from ghalebikesabi2023differentially. Two "ensemble" lines are from ensembles of 5 classifiers. The other two lines show the average accuracy of 5 independently trained classifiers with error bars. Our PE achieves better accuracy across almost all settings with smaller privacy costs.
  • ...and 37 more figures

Theorems & Definitions (6)

  • Theorem 1
  • Theorem 2
  • proof
  • Claim 1
  • proof
  • proof