Heavy-Tailed Privacy: The Symmetric alpha-Stable Privacy Mechanism

TL;DR

The paper addresses the need for a privacy mechanism that delivers pure Differential Privacy while preserving convolution-closure, which the Gaussian mechanism cannot provide. It introduces the Symmetric alpha-Stable (SaS) mechanism, where independent noise components are drawn from a symmetric alpha-stable density $p_{SaS}(x; \alpha, \gamma)$, and proves $\varepsilon$-DP for $\alpha \in [1,2)$. The authors derive how the privacy budget scales with noise and sensitivity, show a closed-form expression for the one-dimensional privacy loss behavior, and quantify the mean absolute distortion via $\mathbb{E}[|Y|] = \frac{2\gamma}{\pi} \Gamma(1-\frac{1}{\alpha})$, illustrating near-Gaussian utility as $\alpha\to 2$. They compare SaS to Laplace and Gaussian mechanisms, demonstrating that SaS can achieve comparable distortion with a strict DP guarantee, making it appealing for privacy-critical applications including local DP and federated learning contexts.

Abstract

With the rapid growth of digital platforms, there is increasing apprehension about how personal data is collected, stored, and used by various entities. These concerns arise from the increasing frequency of data breaches, cyber-attacks, and misuse of personal information for targeted advertising and surveillance. To address these matters, Differential Privacy (DP) has emerged as a prominent tool for quantifying a digital system's level of protection. The Gaussian mechanism is commonly used because the Gaussian density is closed under convolution, and is a common method utilized when aggregating datasets. However, the Gaussian mechanism only satisfies an approximate form of Differential Privacy. In this work, we present and analyze of the Symmetric alpha-Stable (SaS) mechanism. We prove that the mechanism achieves pure differential privacy while remaining closed under convolution. Additionally, we study the nuanced relationship between the level of privacy achieved and the parameters of the density. Lastly, we compare the expected error introduced to dataset queries by the Gaussian and SaS mechanisms. From our analysis, we believe the SaS Mechanism is an appealing choice for privacy-focused applications.

Figures (10)

• Figure 1: In order to protect client identity, Differential Privacy injects noise into the output of a query $f$ on a dataset. This induces a probability density over possible outcomes. A mechanism, $\mathcal{M}_f$, is considered private, if the resulting distributions are essentially the same regardless of the inclusion or exclusion of a single client, shown here in red. Differential Privacy quantifies how much information an adversary is able to gain about the red client.
• Figure 2: Pure-Differential Privacy limits the amount of information an adversary can gain from the outcome of private query. Based on the adversary's initial estimate of the alternative hypothesis, $\textrm{Pr}[\mathcal{D} = \mathcal{D}_1]$, a Differentially Private mechanism bounds the conditional probability given the outcome of the query. Each pair of matching curves represents the lower and upper bound for an adversary's estimate of the alternative hypothesis after observing the outcome of the privacy mechanism. As the privacy budget $\varepsilon$ is increased, the bound of the adversary's updated estimate is increased.
• Figure 3: The family of Symmetric alpha-Stable densities consists of bell shaped densities with varying tail weights determined by the stability parameter $\alpha$. This family of densities is unique because it is the only set of densities that are closed under convolution. When $\alpha=1$, shown in blue $\circ$, the density is known as the Cauchy. When $\alpha=2$, shown in green $\triangle$, the density is known as the Gaussian. No other values of alpha (for the symmetric case) have a known closed form solution, for example $\alpha=1.5$, shown in orange +.
• Figure 4: The real part of the integrand of (\ref{['eqn:stab']}) for $\alpha=1.5$, $\gamma=1$, and $\mu=0$ is an infinitely oscillating function. The value of the stable density with these parameters at the point $x=10$ is the integral of this function on the real line.
• Figure 5: Consider two bell curves, shown here as $p_1$ in blue and $p_2$ in orange, with location parameters $\mu_1 > \mu_2$ respectively. Given a point $x^* > \mu_1$, denote by $d$ the distance between the curves at $x^*$: $d := p_1(x^*) - p_2(x^*)$. Shifting the distribution $p_1$ to the right by some positive value $\epsilon$, gives the curve $p'_1$ shown as a dotted line. By Lemma \ref{['lem:bell']}, the distance $d' := p'_1(x^*+\epsilon) - p_2(x^*+\epsilon)$ is necessarily larger than $d$.

Theorems & Definitions (32)

• Definition 1
• Definition 2
• Definition 3
• Definition 4
• Definition 5
• Theorem 6
• Definition 7
• Definition 8
• Definition 9
• Definition 10