Private Lossless Multiple Release

TL;DR

This paper advances differential privacy by enabling lossless multiple releases across multiple budgets, allowing analysts with different trust levels to receive correlated releases whose collective privacy equals the maximum budget in any subset. It develops a unifying convolution-preorder framework that applies to Gaussian, Laplace, and Poisson noise, providing online sampling procedures that preserve identical distributions to single-release baselines while maintaining the max-budget privacy bound under arbitrary release order. The main technical contribution is a meta theorem showing lossless multiple release for independent additive-noise mechanisms, plus concrete instantiations for Gaussian, Poisson, and Laplace, including efficient methods for sparse histograms and factorization mechanisms via invertible post-processing. The practical impact is a versatile toolkit for multi-level privacy in real-world deployments (e.g., internal/external analysts, data markets, and federated settings) with provable privacy guarantees and favorable utility, demonstrated by empirical evaluations. Open questions include extending lossless release to mechanisms beyond additive noise (e.g., exponential mechanism) and implementing finite-precision variants (e.g., discrete Gaussian) suitable for deployment.

Abstract

Koufogiannis et al. (2016) showed a $\textit{gradual release}$ result for Laplace noise-based differentially private mechanisms: given an $\varepsilon$-DP release, a new release with privacy parameter $\varepsilon' > \varepsilon$ can be computed such that the combined privacy loss of both releases is at most $\varepsilon'$ and the distribution of the latter is the same as a single release with parameter $\varepsilon'$. They also showed gradual release techniques for Gaussian noise, later also explored by Whitehouse et al. (2022). In this paper, we consider a more general $\textit{multiple release}$ setting in which analysts hold private releases with different privacy parameters corresponding to different access/trust levels. These releases are determined one by one, with privacy parameters in arbitrary order. A multiple release is $\textit{lossless}$ if having access to a subset $S$ of the releases has the same privacy guarantee as the least private release in $S$, and each release has the same distribution as a single release with the same privacy parameter. Our main result is that lossless multiple release is possible for a large class of additive noise mechanisms. For the Gaussian mechanism we give a simple method for lossless multiple release with a short, self-contained analysis that does not require knowledge of the mathematics of Brownian motion. We also present lossless multiple release for the Laplace and Poisson mechanisms. Finally, we consider how to efficiently do gradual release of sparse histograms, and present a mechanism with running time independent of the number of dimensions.

Figures (2)

• Figure 1: The idea behind lossless multiple release. For concreteness we consider additive noise mechanism and zero-concentrated differential privacy. Each $Y_{\rho_i}$ denotes a noisy estimate, and to release a new estimate with $\rho > 0$, we can combine the adjacent estimates for $\rho_k$ and $\rho_{k+1}$ together with some fresh noise to obtain a new release $Y_\rho$ that is exactly $\rho$-zCDP. Note that these estimates do not need to be strictly increasing (or decreasing) but can be released in any order. Furthermore, releasing any subset of these estimates is exactly $\max(S)$-zCDP, where $S$ is the set of privacy parameters.
• Figure 2: Accuracy comparison of multiple uncorrelated releases compared to lossless multiple release. Budgets are spaced evenly on a logarithmic scale between $\rho=0.001$ and $\rho=5$ on the x-axis. Creating independent releases with a denser set of privacy parameters comes at the cost of increased variance. In the lossless setting we get the best possible variance with no bound on the number of releases.

Theorems & Definitions (63)

• Definition 1.1: Convolution preorder
• Theorem 1.2: Meta theorem, informal
• Lemma 3.1
• proof
• Lemma 3.2
• proof
• Lemma 3.3
• proof : Proof sketch.
• Definition 3.4: Lossless multiple release
• Corollary 3.5