Laplace Transform Interpretation of Differential Privacy

TL;DR

It is shown that recognizing the bare form expression of Differential Privacy notions as a Laplace transform unlocks a new way to reason about DP properties by exploiting the duality between time and frequency domains.

Abstract

We introduce a set of useful expressions of Differential Privacy (DP) notions in terms of the Laplace transform of the privacy loss distribution. Its bare form expression appears in several related works on analyzing DP, either as an integral or an expectation. We show that recognizing the expression as a Laplace transform unlocks a new way to reason about DP properties by exploiting the duality between time and frequency domains. Leveraging our interpretation, we connect the $(q, ρ(q))$-Rényi DP curve and the $(ε, δ(ε))$-DP curve as being the Laplace and inverse-Laplace transforms of one another. This connection shows that the Rényi divergence is well-defined for complex orders $q = γ+ i ω$. Using our Laplace transform-based analysis, we also prove an adaptive composition theorem for $(ε, δ)$-DP guarantees that is exactly tight (i.e., matches even in constants) for all values of $ε$. Additionally, we resolve an issue regarding symmetry of $f$-DP on subsampling that prevented equivalence across all functional DP notions.

Figures (5)

• Figure 1: Comparison between the indistinguishability characteristic of Gaussian mechanism of \ref{['thm:gaussian_privacy']} ($\kappa = \Vert\mu - \mu'\Vert^2/2\sigma^2 = \varepsilon^2 / 2$) and randomized response mechanism defined in \ref{['thm:rr_privacy_profile']} (with $\delta = 0$). This figure visualizes the singularities at $q = 0$ and $q = 1$ that exists for the Laplace transform $\mathcal{B}\left\{\delta_{P|Q}\right\}(1-q)$ disappears for Rényi divergence $\mathrm{R}_{q}\left(P\middle\Vert Q\right)$, which is an effect of the replica trick unfolding. This figure also demonstrates that neither the dominance $\mathrm{R}_{q}\left(P_1\middle\Vert Q_1\right) \leq \mathrm{R}_{q}\left(P_2\middle\Vert Q_2\right)$ for all $q > 1$, nor the dominance $\mathcal{B}\left\{\delta_{P_1|Q_1}\right\}(1-q) \leq \mathcal{B}\left\{\delta_{P_2|Q_2}\right\}(1-q)$ for all $q \in \mathbb{R} \setminus \{0, 1\}$ is enough to bound $\delta_{P_1|Q_1}(\varepsilon) \leq \delta_{P_2|Q_2}(\varepsilon)$ at all $\varepsilon \in \mathbb{R}$. Additionally, the black dotted line in the rightmost plot shows that even the tightestconversion on the Rényi curve considering only real orders $q > 1$ fails to characterize its own privacy profile.
• Figure 2: Comparison of $(\varepsilon, \delta)$-DP bounds from various composition theorems for $k$-fold composition of a $(0.1, 10^{-8})$-DP point guarantee, with the budget constraint $\delta < 10^{-6}$. The spikes in the right plot, showing exact $\delta < 10^{-6}$ values from kairouz2015composition, occur because, out of a set of $\lfloor k/2 \rfloor$ DP point guarantees by their result, we select the smallest $\varepsilon$ corresponding to the largest $\delta < 10^{-6}$ in the set, which fluctuates as $k$ increases.
• Figure 3: Visualization of the four functional notions of DP, namely privacy profile $\delta_{P|Q}(\varepsilon)$ as a function of $\varepsilon$, the generalized density function of privacy loss distribution $\mathrm{PLD}(P\Vert Q)$, the Rényi divergence $\mathrm{R}_{q}\left(P\middle\Vert Q\right)$ as a function of order $q$, and the trade-off function $f_{P|Q}(\alpha)$ for hypothesis testing between $P$ and $Q$. We also provide the reversal theorems for each of the plots.
• Figure 4: Comparison of $(\varepsilon, \delta)$-DP bounds between numerical accountants and our \ref{['corr:approxdp_composition_homo']} for $100$-fold composition of a $(0.1, 10^{-10})$-DP point guarantee, with the budget constraint $\delta < 10^{-8}$. Note that at $k=100$, the constraint on $\delta$ cannot be satisfied and so the corresponding $\varepsilon = \infty$ at that value. We note that at smaller values of $k$, numerical accountants can sometimes over exceed the budget constraints on $\delta$. Additionally, the gap for $\varepsilon$ between our exact bound and those approximated by numerical accountant tend to be of the order $\approx 10^{-7}$ for Google's PLDAccountant and $\approx 10^{-3}$ for Microsoft's PRVAccountant.
• Figure :

Theorems & Definitions (10)

• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof