Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Encode, Shuffle, Analyze Privacy Revisited: Formalizations and Empirical Evaluation

Úlfar Erlingsson, Vitaly Feldman, Ilya Mironov, Ananth Raghunathan, Shuang Song, Kunal Talwar, Abhradeep Thakurta

TL;DR

<3-5 sentence high-level summary> The paper reexamines the Encode-Shuffle-Analyze (ESA) framework to provide formal guidelines for privacy-preserving reporting with anonymity. It introduces removal LDP, attribute and report fragmentation, and a practical attack model to clarify privacy/utility tradeoffs, demonstrating strong central DP guarantees when reports are anonymized. Through extensive experiments on heavy-tailed and flat-tailed data, as well as training deep models with ESA-based privacy, the work shows high utility is achievable under anonymity, while cautioning that sketch-based encodings require careful tuning and more formal analysis. The results offer practitioners concrete methods for deploying private statistical reporting and shed light on the practical limits of certain ESA components like Crowd IDs and sketches.</p>

Abstract

Recently, a number of approaches and techniques have been introduced for reporting software statistics with strong privacy guarantees. These range from abstract algorithms to comprehensive systems with varying assumptions and built upon local differential privacy mechanisms and anonymity. Based on the Encode-Shuffle-Analyze (ESA) framework, notable results formally clarified large improvements in privacy guarantees without loss of utility by making reports anonymous. However, these results either comprise of systems with seemingly disparate mechanisms and attack models, or formal statements with little guidance to practitioners. Addressing this, we provide a formal treatment and offer prescriptive guidelines for privacy-preserving reporting with anonymity. We revisit the ESA framework with a simple, abstract model of attackers as well as assumptions covering it and other proposed systems of anonymity. In light of new formal privacy bounds, we examine the limitations of sketch-based encodings and ESA mechanisms such as data-dependent crowds. We also demonstrate how the ESA notion of fragmentation (reporting data aspects in separate, unlinkable messages) improves privacy/utility tradeoffs both in terms of local and central differential-privacy guarantees. Finally, to help practitioners understand the applicability and limitations of privacy-preserving reporting, we report on a large number of empirical experiments. We use real-world datasets with heavy-tailed or near-flat distributions, which pose the greatest difficulty for our techniques; in particular, we focus on data drawn from images that can be easily visualized in a way that highlights reconstruction errors. Showing the promise of the approach, and of independent interest, we also report on experiments using anonymous, privacy-preserving reporting to train high-accuracy deep neural networks on standard tasks---MNIST and CIFAR-10.

Encode, Shuffle, Analyze Privacy Revisited: Formalizations and Empirical Evaluation

TL;DR

<3-5 sentence high-level summary> The paper reexamines the Encode-Shuffle-Analyze (ESA) framework to provide formal guidelines for privacy-preserving reporting with anonymity. It introduces removal LDP, attribute and report fragmentation, and a practical attack model to clarify privacy/utility tradeoffs, demonstrating strong central DP guarantees when reports are anonymized. Through extensive experiments on heavy-tailed and flat-tailed data, as well as training deep models with ESA-based privacy, the work shows high utility is achievable under anonymity, while cautioning that sketch-based encodings require careful tuning and more formal analysis. The results offer practitioners concrete methods for deploying private statistical reporting and shed light on the practical limits of certain ESA components like Crowd IDs and sketches.</p>

Abstract

Recently, a number of approaches and techniques have been introduced for reporting software statistics with strong privacy guarantees. These range from abstract algorithms to comprehensive systems with varying assumptions and built upon local differential privacy mechanisms and anonymity. Based on the Encode-Shuffle-Analyze (ESA) framework, notable results formally clarified large improvements in privacy guarantees without loss of utility by making reports anonymous. However, these results either comprise of systems with seemingly disparate mechanisms and attack models, or formal statements with little guidance to practitioners. Addressing this, we provide a formal treatment and offer prescriptive guidelines for privacy-preserving reporting with anonymity. We revisit the ESA framework with a simple, abstract model of attackers as well as assumptions covering it and other proposed systems of anonymity. In light of new formal privacy bounds, we examine the limitations of sketch-based encodings and ESA mechanisms such as data-dependent crowds. We also demonstrate how the ESA notion of fragmentation (reporting data aspects in separate, unlinkable messages) improves privacy/utility tradeoffs both in terms of local and central differential-privacy guarantees. Finally, to help practitioners understand the applicability and limitations of privacy-preserving reporting, we report on a large number of empirical experiments. We use real-world datasets with heavy-tailed or near-flat distributions, which pose the greatest difficulty for our techniques; in particular, we focus on data drawn from images that can be easily visualized in a way that highlights reconstruction errors. Showing the promise of the approach, and of independent interest, we also report on experiments using anonymous, privacy-preserving reporting to train high-accuracy deep neural networks on standard tasks---MNIST and CIFAR-10.

Paper Structure

This paper contains 22 sections, 14 theorems, 22 equations, 2 figures, 13 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Statistical Reporting with Privacy, in Practice
  3. Practical Experiments, Primitives, and Attack Models
  4. Summary of Contributions
  5. Definitions and Assumptions
  6. Local Differential Privacy and Removal vs. Replacement
  7. Attributes, Encodings, and Fragments of Reports
  8. Anonymity and Attack Models
  9. Central Differential Privacy and Amplification by Shuffling
  10. Histograms via Attribute Fragmenting
  11. Report Fragmenting
  12. Crowds and crowd IDs
  13. Machine Learning in the ESA framework
  14. Experimental Evaluation
  15. A Dataset with a Heavy-Hitter Powerlaw Distribution
  16. ...and 7 more sections

Key Result

Lemma 2.5

For $\delta \in [0,1]$ and $\varepsilon_\ell \leq \log(n/\log(1/\delta))/2$, the output of a shuffler that shuffles $n$ reports that are outputs of a $\varepsilon_\ell$-DP local randomizers satisfy $(\varepsilon, \delta)$-DP where $\varepsilon = O\left((e^{\varepsilon_\ell}-1)\sqrt{\log(1/\delta)/n}

Figures (2)

  • Figure 1: A differentially-private view of the NYC smartphone-location data published by the New York Times in 2018 nytimesdata. Anonymous, randomized location reports allow high accuracy with a strong central differential privacy guarantee ($\varepsilon_c=0.5$) and a weaker local guarantee ($\varepsilon_{\ell^{\infty}}\approx 12$) that still provides uncertainty even if all parties collude and break report anonymity.
  • Figure 2: Accuracy vs iterations tradeoff on various data sets, with local differential privacy-per-record-per-iteration $\varepsilon_{\ell e}=1.8$ for CIFAR-10, and $\varepsilon_{\ell e}=1.9$ for MNIST and Fashion-MNIST. The plots are over at least ten independent runs.

Theorems & Definitions (26)

  • Definition 2.1: Replacement $(\varepsilon,\delta)$-DP odo
  • Definition 2.2: Replacement LDP
  • Definition 2.3: Generalized removal $(\varepsilon,\delta)$-DP
  • Definition 2.4: Removal LDP
  • Lemma 2.5
  • Lemma 2.6
  • Theorem 2.7: Advanced Composition Theorem bun2016concentrated
  • Theorem 3.1: Privacy guarantee
  • Theorem 3.2: Utility/privacy tradeoff
  • Theorem 4.1
  • ...and 16 more