Infinitely Divisible Noise for Differential Privacy: Nearly Optimal Error in the High $\varepsilon$ Regime
Charlie Harrison, Pasin Manurangsi
TL;DR
This work advances distributed differential privacy by introducing two infinitely divisible, discrete noise mechanisms—Generalized Discrete Laplace (GDL) and Multi-Scale Discrete Laplace (MSDLap)—and proving they achieve order-optimal MSE in high-$\varepsilon$ regimes, with $\mathrm{MSE}=O(\Delta^3 e^{-\varepsilon})$ for GDL and $\mathrm{MSE}=O(\min(\Delta^3 e^{-\varepsilon}, \Delta^2 e^{-2\varepsilon/3}))$ for MSDLap; through a discrete-to-continuous transformation they obtain a continuous, infinitely divisible mechanism with the optimal $\mathrm{MSE}=O(\Delta^2 e^{-2\varepsilon/3})$. The paper also provides an exact, efficient sampler for distributed MSDLap, generalizes MSDLap to tune accuracy via a parameter $r$, and demonstrates a practical application by improving a multi-message shuffle-DP protocol to achieve $\mathrm{MSE}=O(e^{-2\varepsilon/3})$ with near-minimal message complexity. By showing that infinite divisibility does not incur an utility gap in the pure-DP setting and outperforming existing baselines like Arete in the continuous case, the results offer both theoretical and practical impact for secure aggregation and distributed privacy-preserving querying. The work also lays groundwork for broader use of infinite-divisible noise in distributed private computation and raises open questions about constants and alternative noise-spread strategies.
Abstract
Differential privacy (DP) can be achieved in a distributed manner, where multiple parties add independent noise such that their sum protects the overall dataset with DP. A common technique here is for each party to sample their noise from the decomposition of an infinitely divisible distribution. We analyze two mechanisms in this setting: 1) the generalized discrete Laplace (GDL) mechanism, whose distribution (which is closed under summation) follows from differences of i.i.d. negative binomial shares, and 2) the multi-scale discrete Laplace (MSDLap) mechanism, a novel mechanism following the sum of multiple i.i.d. discrete Laplace shares at different scales. For $\varepsilon \geq 1$, our mechanisms can be parameterized to have $O\left(Δ^3 e^{-\varepsilon}\right)$ and $O\left(\min\left(Δ^3 e^{-\varepsilon}, Δ^2 e^{-2\varepsilon/3}\right)\right)$ MSE, respectively, where $Δ$ denote the sensitivity; the latter bound matches known optimality results. We also show a transformation from the discrete setting to the continuous setting, which allows us to transform both mechanisms to the continuous setting and thereby achieve the optimal $O\left(Δ^2 e^{-2\varepsilon / 3}\right)$ MSE. To our knowledge, these are the first infinitely divisible additive noise mechanisms that achieve order-optimal MSE under pure DP, so our work shows formally there is no separation in utility when query-independent noise adding mechanisms are restricted to infinitely divisible noise. For the continuous setting, our result improves upon the Arete mechanism from [Pagh and Stausholm, ALT 2022] which gives an MSE of $O\left(Δ^2 e^{-\varepsilon/4}\right)$. Furthermore, we give an exact sampler tuned to efficiently implement the MSDLap mechanism, and we apply our results to improve a state of the art multi-message shuffle DP protocol in the high $\varepsilon$ regime.