Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robustness Implies Privacy in Statistical Estimation

Samuel B. Hopkins, Gautam Kamath, Mahbod Majid, Shyam Narayanan

TL;DR

This work揭 explores the fundamental link between robustness to adversarial perturbations and differential privacy in high-dimensional statistics. It introduces a black-box reduction that converts robust estimators into private estimators, enabling near-optimal tradeoffs among sample complexity, accuracy, and privacy for core tasks like mean and covariance estimation of high-dimensional Gaussians; in some settings, this reduction yields polynomial-time private algorithms via SoS-based robust methods. The paper then develops a general private sampling framework that can implement the exponential mechanism with imperfect score oracles, and applies it to mean estimation under DP and to preconditioning the Gaussian to achieve private covariance estimation, all while achieving near-optimal corruption resilience. In particular, it provides the first polynomial-time private estimators for Gaussians in TV distance, as well as nearly optimal DP guarantees for mean and covariance estimation under both pure and approximate DP, using SoS-based robust estimators and careful volume/sampling analysis. The results collectively hint at an algorithmic equivalence between robustness and privacy in “nice” high-dimensional estimation problems, with strong implications for private statistical learning in Gaussian settings and beyond.

Abstract

We study the relationship between adversarial robustness and differential privacy in high-dimensional algorithmic statistics. We give the first black-box reduction from privacy to robustness which can produce private estimators with optimal tradeoffs among sample complexity, accuracy, and privacy for a wide range of fundamental high-dimensional parameter estimation problems, including mean and covariance estimation. We show that this reduction can be implemented in polynomial time in some important special cases. In particular, using nearly-optimal polynomial-time robust estimators for the mean and covariance of high-dimensional Gaussians which are based on the Sum-of-Squares method, we design the first polynomial-time private estimators for these problems with nearly-optimal samples-accuracy-privacy tradeoffs. Our algorithms are also robust to a nearly optimal fraction of adversarially-corrupted samples.

Robustness Implies Privacy in Statistical Estimation

TL;DR

This work揭 explores the fundamental link between robustness to adversarial perturbations and differential privacy in high-dimensional statistics. It introduces a black-box reduction that converts robust estimators into private estimators, enabling near-optimal tradeoffs among sample complexity, accuracy, and privacy for core tasks like mean and covariance estimation of high-dimensional Gaussians; in some settings, this reduction yields polynomial-time private algorithms via SoS-based robust methods. The paper then develops a general private sampling framework that can implement the exponential mechanism with imperfect score oracles, and applies it to mean estimation under DP and to preconditioning the Gaussian to achieve private covariance estimation, all while achieving near-optimal corruption resilience. In particular, it provides the first polynomial-time private estimators for Gaussians in TV distance, as well as nearly optimal DP guarantees for mean and covariance estimation under both pure and approximate DP, using SoS-based robust estimators and careful volume/sampling analysis. The results collectively hint at an algorithmic equivalence between robustness and privacy in “nice” high-dimensional estimation problems, with strong implications for private statistical learning in Gaussian settings and beyond.

Abstract

We study the relationship between adversarial robustness and differential privacy in high-dimensional algorithmic statistics. We give the first black-box reduction from privacy to robustness which can produce private estimators with optimal tradeoffs among sample complexity, accuracy, and privacy for a wide range of fundamental high-dimensional parameter estimation problems, including mean and covariance estimation. We show that this reduction can be implemented in polynomial time in some important special cases. In particular, using nearly-optimal polynomial-time robust estimators for the mean and covariance of high-dimensional Gaussians which are based on the Sum-of-Squares method, we design the first polynomial-time private estimators for these problems with nearly-optimal samples-accuracy-privacy tradeoffs. Our algorithms are also robust to a nearly optimal fraction of adversarially-corrupted samples.
Paper Structure (72 sections, 72 theorems, 130 equations, 2 tables)

This paper contains 72 sections, 72 theorems, 130 equations, 2 tables.

Table of Contents

  1. Introduction
  2. Results
  3. Learning High-Dimensional Gaussian Distributions in TV Distance
  4. Estimating the Mean of a Subgaussian Distribution
  5. Related Work
  6. Techniques
  7. Black-Box Reduction from Robustness to Privacy
  8. Algorithms
  9. Robustness to Privacy, Algorithmically
  10. Preliminaries
  11. A General Private Sampling Algorithm
  12. Sampling and volume computation with an imperfect oracle
  13. Proof of \ref{['thm:pure_dp_general_main']}
  14. Proof of \ref{['thm:approx_dp_general_main']}
  15. Estimating the Mean of a Gaussian
  16. ...and 57 more sections

Key Result

Theorem 1.3

Assume that $0 < \alpha, \beta, \varepsilon < 1$, $0 < \eta < \eta^*$ for some absolute constant $\eta^*$, and $K, R > 1$. There is a polynomial-time $(\varepsilon,0)$-DP algorithm with the following guarantees for every $d \in \mathbb{N}$ and every $\mu \in \mathbb{R}^d, \Sigma \in \mathbb{R}^{d \t

Theorems & Definitions (157)

  • Definition 1.1: Differential Privacy (DP) DworkMNS06DworkKMMN06
  • Definition 1.2: Strong Contamination Model
  • Theorem 1.3: Learning Arbitrary Gaussians, Pure DP, Subsection \ref{['subsec:main_stuff']}
  • Theorem 1.4: Learning Arbitrary Gaussians, $(\varepsilon,\delta)$-DP, Subsection \ref{['subsec:main_stuff']}
  • Theorem 1.5: Estimating the Mean of a Spherical Subgaussian Distribution, Theorem \ref{['thm:gaussian-mean-main']}
  • Lemma 2.1
  • proof
  • Lemma 2.2
  • Claim 2.3
  • proof
  • ...and 147 more