Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Estimation of a Gaussian Mean with Local Differential Privacy

Nikita P. Kalinin, Lukas Steinberger

TL;DR

This work solves the Gaussian mean estimation problem under local differential privacy by identifying the sign mechanism with randomized response on $\mathrm{sgn}(X-\theta)$ as optimally informative in the high-privacy regime ($\epsilon\le 1.04$). To overcome the dependency on the unknown $\theta$, the authors develop a two-stage estimator that first privately estimates $\theta$ and then applies the sign-based mechanism with the estimated parameter, achieving asymptotically efficient variance. They derive a complete proof via a discrete approximation, a linear-programming reformulation, and a duality argument, establishing that the sign mechanism attains the maximal Fisher-information and thus the minimal asymptotic variance among all $\epsilon$-private mechanisms in this regime. The results extend to known-variance settings with a simple rescaling and demonstrate practical gains through simulations, while discussing limitations for larger privacy budgets and higher-dimensional problems. The work provides a precise, closed-form solution to an important LDP estimation problem and offers a concrete, tunable procedure for efficient private mean estimation.

Abstract

In this paper we study the problem of estimating the unknown mean $θ$ of a unit variance Gaussian distribution in a locally differentially private (LDP) way. In the high-privacy regime ($ε\le 1$), we identify an optimal privacy mechanism that minimizes the variance of the estimator asymptotically. Our main technical contribution is the maximization of the Fisher-Information of the sanitized data with respect to the local privacy mechanism $Q$. We find that the exact solution $Q_{θ,ε}$ of this maximization is the sign mechanism that applies randomized response to the sign of $X_i-θ$, where $X_1,\dots, X_n$ are the confidential iid original samples. However, since this optimal local mechanism depends on the unknown mean $θ$, we employ a two-stage LDP parameter estimation procedure which requires splitting agents into two groups. The first $n_1$ observations are used to consistently but not necessarily efficiently estimate the parameter $θ$ by $\tildeθ_{n_1}$. Then this estimate is updated by applying the sign mechanism with $\tildeθ_{n_1}$ instead of $θ$ to the remaining $n-n_1$ observations, to obtain an LDP and efficient estimator of the unknown mean.

Efficient Estimation of a Gaussian Mean with Local Differential Privacy

TL;DR

This work solves the Gaussian mean estimation problem under local differential privacy by identifying the sign mechanism with randomized response on as optimally informative in the high-privacy regime (). To overcome the dependency on the unknown , the authors develop a two-stage estimator that first privately estimates and then applies the sign-based mechanism with the estimated parameter, achieving asymptotically efficient variance. They derive a complete proof via a discrete approximation, a linear-programming reformulation, and a duality argument, establishing that the sign mechanism attains the maximal Fisher-information and thus the minimal asymptotic variance among all -private mechanisms in this regime. The results extend to known-variance settings with a simple rescaling and demonstrate practical gains through simulations, while discussing limitations for larger privacy budgets and higher-dimensional problems. The work provides a precise, closed-form solution to an important LDP estimation problem and offers a concrete, tunable procedure for efficient private mean estimation.

Abstract

In this paper we study the problem of estimating the unknown mean of a unit variance Gaussian distribution in a locally differentially private (LDP) way. In the high-privacy regime (), we identify an optimal privacy mechanism that minimizes the variance of the estimator asymptotically. Our main technical contribution is the maximization of the Fisher-Information of the sanitized data with respect to the local privacy mechanism . We find that the exact solution of this maximization is the sign mechanism that applies randomized response to the sign of , where are the confidential iid original samples. However, since this optimal local mechanism depends on the unknown mean , we employ a two-stage LDP parameter estimation procedure which requires splitting agents into two groups. The first observations are used to consistently but not necessarily efficiently estimate the parameter by . Then this estimate is updated by applying the sign mechanism with instead of to the remaining observations, to obtain an LDP and efficient estimator of the unknown mean.
Paper Structure (22 sections, 7 theorems, 81 equations, 2 figures)

This paper contains 22 sections, 7 theorems, 81 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Proof of main results
  4. Proof of Theorem \ref{['thm:main']}
  5. Discrete approximation
  6. Evaluating the quantized objective function
  7. Reformulation as a Linear Program
  8. Guessing a dual solution
  9. One-Stage Variance
  10. General variance
  11. Experiments
  12. 3-Stage Mechanism
  13. Discussion
  14. Appendix
  15. Proof of Theorem \ref{['thm:efficiency']}
  16. ...and 7 more sections

Key Result

Theorem 1

If $\epsilon\le 1.04$ then for all $\theta\in{\mathbb R}$ and all $Q\in\mathcal{Q}_\epsilon$. In particular, the sign mechanism solves eq:Opt.

Figures (2)

  • Figure 1: Left panel: Scaled MSE of the private estimator as a function of the number $n_1$ of first-stage samples. Right panel: Scaled MSE of the private estimator as a function of the initial guess $\theta_0$.
  • Figure 2: The plot compares the performance of our enhanced 3-stage mechanism with the original mechanism from Jos19, including a tuned version in which we adjusted the number of samples allocated to the first stage. Additionally, it is compared with liu2023online, which has been enhanced by a preliminary stage from Jos19. The horizontal lines represent the theoretical variance limits for each mechanism.

Theorems & Definitions (10)

  • Theorem 1
  • Theorem 2
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • Lemma 4
  • proof
  • Theorem 3