Some Constructions of Private, Efficient, and Optimal $K$-Norm and Elliptic Gaussian Noise

TL;DR

The paper develops practical private noise mechanisms tailored to problem-specific sensitivity spaces for sums, counts, and votes. It provides efficient samplers for the optimal $K$-norm mechanisms in $O(d^2)$-style time and derives closed-form, easily-sampled ellipses for elliptic Gaussian noise, improving accuracy over spherical noise in many settings. A key message is that analyzing the sensitivity space and exploiting geometry can yield substantial privacy-utility gains, including parallelizable noise generation for large-scale data. Together, these results offer a viable path to private additive noise that is both faster and more accurate than generic convex-body sampling or semidefinite-program-based ellipse computations.

Abstract

Differentially private computation often begins with a bound on some $d$-dimensional statistic's $\ell_p$ sensitivity. For pure differential privacy, the $K$-norm mechanism can improve on this approach using a norm tailored to the statistic's sensitivity space. Writing down a closed-form description of this optimal norm is often straightforward. However, running the $K$-norm mechanism reduces to uniformly sampling the norm's unit ball; this ball is a $d$-dimensional convex body, so general sampling algorithms can be slow. Turning to concentrated differential privacy, elliptic Gaussian noise offers similar improvement over spherical Gaussian noise. Once the shape of this ellipse is determined, sampling is easy; however, identifying the best such shape may be hard. This paper solves both problems for the simple statistics of sum, count, and vote. For each statistic, we provide a sampler for the optimal $K$-norm mechanism that runs in time $\tilde O(d^2)$ and derive a closed-form expression for the optimal shape of elliptic Gaussian noise. The resulting algorithms all yield meaningful accuracy improvements while remaining fast and simple enough to be practical. More broadly, we suggest that problem-specific sensitivity space analysis may be an overlooked tool for private additive noise.

Figures (2)

• Figure 1: Mean squared $\ell_2$ error ratios. The privacy parameter $\varepsilon$ or $\rho$ controls the scaling of a sample from the induced norm ball ($K$-norm mechanism) or ellipse (elliptic Gaussian noise), so we simply compare expected sample magnitudes for the underlying shapes. For the $K$-norm mechanism (left), we evaluate Sum and Count with dimension $d=50$ and varying contribution bound $k$. We also evaluate Vote, varying $d$ up to $d=50$ (note that Vote does not have a $k$ parameter). Each point compares to the best $\ell_p$ ball at the current parameter over 1,000 trials. For elliptic Gaussian noise (right), we compare to the minimum $\ell_2$ ball, fixing $d=1,000$ and varying $k$ for Count and varying $d$ up to $d=1,000$ for Vote, using closed-form expressions for the expected squared $\ell_2$ norm of a sample from the ellipse or ball in question. The Count ellipse plot covers $k \leq d/2$ because its minimal ellipse result only holds for this sparse-contribution setting. Throughout, a value $< 1$ means our algorithm is better. See Github G24 for simulation code.
• Figure 2: Left: $R_{3,2}$ is the shaded region of the cube. Center: $B_{\mathsf{count}}$, $k=2$; $R_{3,2}$ reappears in the upper right corner. Right: $B_{\mathsf{vote}}$; $CH(P_3)$ is a regular polytope, but this is not true for general $d$.

Theorems & Definitions (132)

• Theorem 1.4: Informal
• Theorem 1.5: Informal
• Definition 2.1: DMNS06BS16
• Lemma 2.2: HT10
• Lemma 2.3: Remark 4.2 HT10
• Definition 2.4: KN16AS21
• Lemma 2.5
• Lemma 2.6: Theorem 3.19 AS21
• Lemma 2.7: Adapted From NTZ13NT23
• Lemma 2.8