Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lower Bounds for Private Estimation of Gaussian Covariance Matrices under All Reasonable Parameter Regimes

Victor S. Portella, Nick Harvey

TL;DR

This paper studies the problem of estimating the covariance of a Gaussian under $(\varepsilon,\delta)$-DP and proves a matching $n=\Omega\left(\frac{d^{2}}{\varepsilon\alpha}\right)$ lower bound for well-conditioned covariances, thereby closing a gap in the parameter regimes where previous work had gaps. The authors extend the score-attack fingerprinting framework to non-i.i.d. priors by leveraging the Stein-Haff identity, and they design a natural prior over $\Sigma$ via a normalized Wishart distribution with $D=2d$ that yields robust, polylog-free lower bounds. The analysis combines a lower bound on a correlation statistic (via Stein-Haff) with an upper bound derived from DP properties and tail bounds, showing that private covariance estimation requires essentially the same sample complexity as the non-private setting up to privacy terms. A key technical contribution is handling non-independent parameters in the score-attack approach, enabling lower bounds for a broad class of priors and parameter regimes. The results provide a unified, rigorous view of DP covariance estimation lower bounds and offer a methodology that may extend to other parameter estimation problems under privacy constraints.

Abstract

We prove lower bounds on the number of samples needed to privately estimate the covariance matrix of a Gaussian distribution. Our bounds match existing upper bounds in the widest known setting of parameters. Our analysis relies on the Stein-Haff identity, an extension of the classical Stein's identity used in previous fingerprinting lemma arguments.

Lower Bounds for Private Estimation of Gaussian Covariance Matrices under All Reasonable Parameter Regimes

TL;DR

This paper studies the problem of estimating the covariance of a Gaussian under -DP and proves a matching lower bound for well-conditioned covariances, thereby closing a gap in the parameter regimes where previous work had gaps. The authors extend the score-attack fingerprinting framework to non-i.i.d. priors by leveraging the Stein-Haff identity, and they design a natural prior over via a normalized Wishart distribution with that yields robust, polylog-free lower bounds. The analysis combines a lower bound on a correlation statistic (via Stein-Haff) with an upper bound derived from DP properties and tail bounds, showing that private covariance estimation requires essentially the same sample complexity as the non-private setting up to privacy terms. A key technical contribution is handling non-independent parameters in the score-attack approach, enabling lower bounds for a broad class of priors and parameter regimes. The results provide a unified, rigorous view of DP covariance estimation lower bounds and offer a methodology that may extend to other parameter estimation problems under privacy constraints.

Abstract

We prove lower bounds on the number of samples needed to privately estimate the covariance matrix of a Gaussian distribution. Our bounds match existing upper bounds in the widest known setting of parameters. Our analysis relies on the Stein-Haff identity, an extension of the classical Stein's identity used in previous fingerprinting lemma arguments.
Paper Structure (27 sections, 25 theorems, 113 equations)

This paper contains 27 sections, 25 theorems, 113 equations.

Table of Contents

  1. Introduction
  2. Our contributions
  3. Related Work
  4. Covariance Matrix Estimation.
  5. Lower Bounds in DP.
  6. Fingerprinting Techniques.
  7. Beyond i.i.d. Priors in Fingerprinting Lemmas.
  8. Notation
  9. DP Lower Bounds via the Score Attack
  10. Proof Overview
  11. Distribution on $\Sigma$
  12. The normalized Wishart Distribution.
  13. Error of the Mechanism.
  14. Completing the proof of Theorem \ref{['thm:Main']}
  15. Lower Bound on the Correlation Satistics via the Stein-Haff Identity
  16. ...and 12 more sections

Key Result

Theorem 1.1

Let $\mathcal{M} \colon (\mathbb{R}^d)^n \to \mathbb{R}^{d \times d}$ be $(\varepsilon, \delta)$-DP with $\varepsilon \in (0,1)$ and Let $\Sigma$ be a random positive definite matrix in $\mathbb{R}^{d \times d}$ and let $\alpha^2$ be the squared error of $\mathcal{M}$ given by expected squared Frobenius norm error conditioned on the event that $\Sigma$ has eigenvalues in $[0.09, 10]$. Then, there

Theorems & Definitions (25)

  • Theorem 1.1: Main Theorem
  • Theorem 2.1: Framework for the score attack, CaiWZ23a
  • Lemma 2.2: Main Lower Bound
  • Lemma 2.3: Main Upper Bound, Fixed $\Sigma$
  • Lemma 2.4: Upper Bound with Random $\Sigma$
  • Theorem 3.1: Stein-Haff Identity, Haff79a
  • Lemma 3.2
  • Lemma 4.1
  • Theorem 1.1
  • Lemma 2.1
  • ...and 15 more