Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differential Privacy with Higher Utility by Exploiting Coordinate-wise Disparity: Laplace Mechanism Can Beat Gaussian in High Dimensions

Gokularam Muthukrishnan, Sheetal Kalyani

TL;DR

The paper addresses the challenge of achieving tighter privacy-utility trade-offs in high-dimensional queries by introducing independent-but-not-identically-distributed noise (i.n.i.d.) across coordinates via a sensitivity profile. It derives optimal per-coordinate scales for i.n.i.d. Gaussian and Laplace mechanisms under $(\epsilon,\delta)$-DP and $\epsilon$-DP, showing that Laplace can outperform Gaussian when coordinate-wise sensitivities are disparate. Theoretical results include convex-optimal allocations and asymptotic MSE reductions of order $O(\nu^2)$, supported by extensive empirical validation in DP-CD, DP-PCA, and private DL with group clipping. The work demonstrates practical gains and provides guidance for leveraging coordinate disparity to improve DP utility, while noting challenges in estimating sensitivity profiles and potential limits under coordinate correlations.

Abstract

Conventionally, in a differentially private additive noise mechanism, independent and identically distributed (i.i.d.) noise samples are added to each coordinate of the response. In this work, we formally present the addition of noise that is independent but not identically distributed (i.n.i.d.) across the coordinates to achieve tighter privacy-accuracy trade-off by exploiting coordinate-wise disparity in privacy leakage. In particular, we study the i.n.i.d. Gaussian and Laplace mechanisms and obtain the conditions under which these mechanisms guarantee privacy. The optimal choice of parameters that ensure these conditions are derived considering (weighted) mean squared and $\ell_{p}^{p}$-errors as measures of accuracy. Theoretical analyses and numerical simulations demonstrate that the i.n.i.d. mechanisms achieve higher utility for the given privacy requirements compared to their i.i.d. counterparts. One of the interesting observations is that the Laplace mechanism outperforms Gaussian even in high dimensions, as opposed to the popular belief, if the irregularity in coordinate-wise sensitivities is exploited. We also demonstrate how the i.n.i.d. noise can improve the performance in private (a) coordinate descent, (b) principal component analysis, and (c) deep learning with group clipping.

Differential Privacy with Higher Utility by Exploiting Coordinate-wise Disparity: Laplace Mechanism Can Beat Gaussian in High Dimensions

TL;DR

The paper addresses the challenge of achieving tighter privacy-utility trade-offs in high-dimensional queries by introducing independent-but-not-identically-distributed noise (i.n.i.d.) across coordinates via a sensitivity profile. It derives optimal per-coordinate scales for i.n.i.d. Gaussian and Laplace mechanisms under -DP and -DP, showing that Laplace can outperform Gaussian when coordinate-wise sensitivities are disparate. Theoretical results include convex-optimal allocations and asymptotic MSE reductions of order , supported by extensive empirical validation in DP-CD, DP-PCA, and private DL with group clipping. The work demonstrates practical gains and provides guidance for leveraging coordinate disparity to improve DP utility, while noting challenges in estimating sensitivity profiles and potential limits under coordinate correlations.

Abstract

Conventionally, in a differentially private additive noise mechanism, independent and identically distributed (i.i.d.) noise samples are added to each coordinate of the response. In this work, we formally present the addition of noise that is independent but not identically distributed (i.n.i.d.) across the coordinates to achieve tighter privacy-accuracy trade-off by exploiting coordinate-wise disparity in privacy leakage. In particular, we study the i.n.i.d. Gaussian and Laplace mechanisms and obtain the conditions under which these mechanisms guarantee privacy. The optimal choice of parameters that ensure these conditions are derived considering (weighted) mean squared and -errors as measures of accuracy. Theoretical analyses and numerical simulations demonstrate that the i.n.i.d. mechanisms achieve higher utility for the given privacy requirements compared to their i.i.d. counterparts. One of the interesting observations is that the Laplace mechanism outperforms Gaussian even in high dimensions, as opposed to the popular belief, if the irregularity in coordinate-wise sensitivities is exploited. We also demonstrate how the i.n.i.d. noise can improve the performance in private (a) coordinate descent, (b) principal component analysis, and (c) deep learning with group clipping.
Paper Structure (34 sections, 11 theorems, 46 equations, 6 figures, 5 tables, 2 algorithms)

This paper contains 34 sections, 11 theorems, 46 equations, 6 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Prior works
  3. Motivations
  4. Outline of our results
  5. Basic notations
  6. Organization of the paper
  7. Background and i.i.d. noise mechanisms
  8. Additive noise mechanism
  9. Sensitivity:
  10. Equivalent characterization of privacy loss:
  11. Classical Gaussian mechanism:
  12. Classical Laplace mechanism:
  13. Non-identical noise for differential privacy
  14. Non-identical Gaussian noise mechanism
  15. Non-identical Laplace noise mechanism
  16. ...and 19 more sections

Key Result

Lemma 1

The Gaussian mechanism that adds i.i.d. noise sampled from $\mathcal{N}(0,\sigma^2_{})$ to each of the $K$ coordinates of the query response is $(\epsilon,\delta)$-differentially private if and only if where $\Delta_2^{}$ is the $\ell_2^{}$-sensitivity of the query.

Figures (6)

  • Figure 1: Performance of i.i.d. and i.n.i.d. $(\epsilon,\delta)$-DP Gaussian mechanisms under various sensitivity profiles with unit $\ell_2^{}$-sensitivity for $\delta=10^{-6}_{}$.
  • Figure 2: Gini coefficients of various sensitivity profiles having unit $\ell_2^{}$-sensitivity with varying dimension $K$.
  • Figure 3: Performance of i.i.d. and i.n.i.d. $\epsilon$-DP Laplace mechanisms under various sensitivity profiles with unit $\ell_1^{}$-sensitivity.
  • Figure 4: Comparison of $(\epsilon,\delta)$-DP Gaussian mechanism and $\epsilon$-DP Laplace mechanism with varying dimension $K$ for $\epsilon=0.5$ and $\delta=10^{-6}_{}$.
  • Figure 5: Performance of i.n.i.d. noise in DP coordinate descent for $\ell_1^{}$-regularized linear regression on California dataset.
  • ...and 1 more figures

Theorems & Definitions (32)

  • definition 1: dwork2014algorithmic
  • definition 2: Additive noise mechanism
  • definition 3: Sensitivity
  • definition 4: Sensitivity profile
  • remark 1
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • proof
  • remark 2
  • ...and 22 more