Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Agnostic Private Density Estimation for GMMs via List Global Stability

Mohammad Afzali, Hassan Ashtiani, Christopher Liaw

TL;DR

An agnostic variant of the notion of $\textit{list global stability}$ is defined, showing that its existence is sufficient for agnostic private density estimation, and an agnostic list globally stable learner for GMMs is constructed.

Abstract

We consider the problem of private density estimation for mixtures of unrestricted high dimensional Gaussians in the agnostic setting. We prove the first upper bound on the sample complexity of this problem. Previously, private learnability of high dimensional GMMs was only known in the realizable setting [Afzali et al., 2024]. To prove our result, we exploit the notion of $\textit{list global stability}$ [Ghazi et al., 2021b,a] that was originally introduced in the context of private supervised learning. We define an agnostic variant of this definition, showing that its existence is sufficient for agnostic private density estimation. We then construct an agnostic list globally stable learner for GMMs.

Agnostic Private Density Estimation for GMMs via List Global Stability

TL;DR

An agnostic variant of the notion of is defined, showing that its existence is sufficient for agnostic private density estimation, and an agnostic list globally stable learner for GMMs is constructed.

Abstract

We consider the problem of private density estimation for mixtures of unrestricted high dimensional Gaussians in the agnostic setting. We prove the first upper bound on the sample complexity of this problem. Previously, private learnability of high dimensional GMMs was only known in the realizable setting [Afzali et al., 2024]. To prove our result, we exploit the notion of [Ghazi et al., 2021b,a] that was originally introduced in the context of private supervised learning. We define an agnostic variant of this definition, showing that its existence is sufficient for agnostic private density estimation. We then construct an agnostic list globally stable learner for GMMs.
Paper Structure (14 sections, 14 theorems, 20 equations, 4 algorithms)

This paper contains 14 sections, 14 theorems, 20 equations, 4 algorithms.

Table of Contents

  1. Introduction
  2. Private density estimation for GMMs
  3. Our contributions
  4. Paper organization
  5. Preliminaries
  6. Differential privacy
  7. From list global stability to agnostic private density estimation
  8. List globally stable learning of mixture distributions
  9. Agnostic private learning of GMMs
  10. List globally stable learning of Gaussians and their mixtures
  11. Agnostic private learning of GMMs
  12. More on related work
  13. Discussion on stability
  14. Additional facts

Key Result

Theorem 1.4

Let $\mathcal{F}$ be a class of distributions. For any $m,L\in \mathbb{N}$, $\alpha,\beta \in (0,1), C > 1$, if $\mathcal{F}$ is $(C, \frac{\alpha}{3+4C})$-accurate $(m,0.91, L)$-list-globally-stable learnable, then $\mathcal{F}$ is $(\varepsilon,\delta)$-privately $7C$-agnostic $(n,\alpha,\beta)$-l

Theorems & Definitions (36)

  • Definition 1.1: Agnostic Density Estimation
  • Definition 1.2: List Global Stability ghazi2021sampleghazi2021user
  • Definition 1.3: List Global Stability for Agnostic Density Estimation
  • Theorem 1.4: Private agnostic learning via list global stability
  • Theorem 1.5: Private agnostic learning GMMs, informal version of \ref{['thm:main-gmm-agnostic']}
  • Definition 2.1: $\alpha$-cover
  • Definition 2.2: $k$-mixtures
  • Definition 2.3: Unbounded Gaussians
  • Theorem 2.4: Learning finite classes, Theorem 6.3 of devroye2001combinatorial
  • Definition 2.5: $(\varepsilon,\delta)$-Indistinguishable
  • ...and 26 more