Universal Exact Compression of Differentially Private Mechanisms

TL;DR

Results on distributed mean estimation show that PPR consistently gives a better trade-off between communication, accuracy and central differential privacy compared to the coordinate subsampled Gaussian mechanism, while also providing local differential privacy.

Abstract

To reduce the communication cost of differential privacy mechanisms, we introduce a novel construction, called Poisson private representation (PPR), designed to compress and simulate any local randomizer while ensuring local differential privacy. Unlike previous simulation-based local differential privacy mechanisms, PPR exactly preserves the joint distribution of the data and the output of the original local randomizer. Hence, the PPR-compressed privacy mechanism retains all desirable statistical properties of the original privacy mechanism such as unbiasedness and Gaussianity. Moreover, PPR achieves a compression size within a logarithmic gap from the theoretical lower bound. Using the PPR, we give a new order-wise trade-off between communication, accuracy, central and local differential privacy for distributed mean estimation. Experiment results on distributed mean estimation show that PPR consistently gives a better trade-off between communication, accuracy and central differential privacy compared to the coordinate subsampled Gaussian mechanism, while also providing local differential privacy.

Figures (5)

• Figure 1: MSE of distributed mean estimation for PPR and CSGM chen2024privacy for different $\varepsilon$'s.
• Figure 2: MSE of PPR-compressed Laplace mechanism and discrete Laplace mechanism andres2013geo for different $\varepsilon$'s.
• Figure 3: Average running time of PPR applied to a chunk of dimension $d_{\mathrm{chunk}}$, with error bars indicating the interval $T_{\mathrm{chunk}} \pm 2 \sigma_{\mathrm{mean}}$, where $T_{\mathrm{chunk}}$ is the sample mean of the running time, and $\sigma_{\mathrm{mean}}$ is the standard error of the mean (see Footnote \ref{['footnote:stderr']}).
• Figure 4: Average running time (over $20000$ trials), $d_{\mathrm{chunk}}=4$ and $\varepsilon\in [0.06, 10]$, with error bars indicating the interval $T_{\mathrm{chunk}} \pm 2 \sigma_{\mathrm{mean}}$, where $T_{\mathrm{chunk}}$ is the sample mean of the running time, and $\sigma_{\mathrm{mean}}$ is the standard error of the mean.
• Figure 5: The MSE of PPR and CSGM against the compression size in bits, where $\varepsilon$ is chosen from $\{0.25, 0.5, 1.0, 2.0\}$ and compression sizes vary from $25$ to $1000$ bits. Note that parts of the curves for PPR are flat, because a lower compression size is already sufficient for PPR to exactly simulate the best Gaussian mechanism for that value of $\varepsilon$, so a higher compression size than necessary will not affect the result.

Theorems & Definitions (19)

• Definition 3.1: Differential privacy dwork2006calibratingkasiviswanathan2011can
• Definition 3.2: $\varepsilon\cdot d_{\mathcal{X}}$-privacy chatzikokolakis2013broadeningandres2013geo
• Definition 4.1: Poisson functional representation sfrl_transli2021unified
• Proposition 4.2
• Theorem 4.3: Compression size of PPR
• Corollary 4.4: Compression size under $\varepsilon$-LDP
• Theorem 4.5: $\varepsilon$-DP of PPR
• Theorem 4.6: $(\varepsilon,\delta)$-DP of PPR
• Theorem 4.7: Metric privacy of PPR
• Theorem 4.8: Tighter $(\varepsilon,\delta)$-DP of PPR