Covariance-Aware Private Mean Estimation Without Private Covariance Estimation
Gavin Brown, Marco Gaboardi, Adam Smith, Jonathan Ullman, Lydia Zakynthinou
TL;DR
This work tackles the challenge of differentially private mean estimation in high dimensions when the covariance Σ is unknown. It introduces two sample-efficient estimators that achieve Mahalanobis-distance accuracy ||μ̂−μ||_Σ ≤ α with near-optimal dependence on the dimension and privacy parameters: a Tukey-depth based exponential mechanism restricted to high-depth outputs with a private safety test, and an empirically rescaled Gaussian mechanism that privately perturbs the empirical mean using the empirical covariance without releasing Σ, applicable to subgaussian data. The Tukey-depth approach provides robustness to adversarial corruptions and achieves accuracy under Gaussian assumptions with near-optimal sample complexity; the rescaled Gaussian mechanism replaces σ-scaled spherical noise with covariance-aware noise calibrated to Σ̂, plus a private projection to a “good” data-set class to ensure privacy. Together, these results bypass the costly private covariance estimation bottleneck and yield nearly optimal private mean estimation guarantees for Gaussian and subgaussian data, with clear DP analyses and conditions for practical implementation. The work advances practical differential privacy for high-dimensional statistics by eliminating private covariance estimation as a prerequisite and offering two complementary, theory-backed strategies with distinct robustness and distributional assumptions.
Abstract
We present two sample-efficient differentially private mean estimators for $d$-dimensional (sub)Gaussian distributions with unknown covariance. Informally, given $n \gtrsim d/α^2$ samples from such a distribution with mean $μ$ and covariance $Σ$, our estimators output $\tildeμ$ such that $\| \tildeμ- μ\|_Σ \leq α$, where $\| \cdot \|_Σ$ is the Mahalanobis distance. All previous estimators with the same guarantee either require strong a priori bounds on the covariance matrix or require $Ω(d^{3/2})$ samples. Each of our estimators is based on a simple, general approach to designing differentially private mechanisms, but with novel technical steps to make the estimator private and sample-efficient. Our first estimator samples a point with approximately maximum Tukey depth using the exponential mechanism, but restricted to the set of points of large Tukey depth. Its accuracy guarantees hold even for data sets that have a small amount of adversarial corruption. Proving that this mechanism is private requires a novel analysis. Our second estimator perturbs the empirical mean of the data set with noise calibrated to the empirical covariance, without releasing the covariance itself. Its sample complexity guarantees hold more generally for subgaussian distributions, albeit with a slightly worse dependence on the privacy parameter. For both estimators, careful preprocessing of the data is required to satisfy differential privacy.