Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Private Statistical Estimation via Truncation

Manolis Zampetakis, Felix Zhou

TL;DR

This work tackles private statistical estimation when data have unbounded support by introducing a principled truncation framework that bounds sensitivity. By restricting to a carefully chosen survival set and performing DP-SGD on the truncated negative log-likelihood, the authors correct for truncation bias while maintaining strong convergence properties. They prove a uniform convergence result for the empirical likelihood and establish strong convexity on the truncated objective, enabling efficient private estimation of exponential-family parameters. The framework recovers optimal sample complexities for Gaussian mean and covariance estimation and provides a general blueprint for private estimation of unbounded high-dimensional exponential families, with potential broad impact on privacy-preserving data analysis in practice.

Abstract

We introduce a novel framework for differentially private (DP) statistical estimation via data truncation, addressing a key challenge in DP estimation when the data support is unbounded. Traditional approaches rely on problem-specific sensitivity analysis, limiting their applicability. By leveraging techniques from truncated statistics, we develop computationally efficient DP estimators for exponential family distributions, including Gaussian mean and covariance estimation, achieving near-optimal sample complexity. Previous works on exponential families only consider bounded or one-dimensional families. Our approach mitigates sensitivity through truncation while carefully correcting for the introduced bias using maximum likelihood estimation and DP stochastic gradient descent. Along the way, we establish improved uniform convergence guarantees for the log-likelihood function of exponential families, which may be of independent interest. Our results provide a general blueprint for DP algorithm design via truncated statistics.

Private Statistical Estimation via Truncation

TL;DR

This work tackles private statistical estimation when data have unbounded support by introducing a principled truncation framework that bounds sensitivity. By restricting to a carefully chosen survival set and performing DP-SGD on the truncated negative log-likelihood, the authors correct for truncation bias while maintaining strong convergence properties. They prove a uniform convergence result for the empirical likelihood and establish strong convexity on the truncated objective, enabling efficient private estimation of exponential-family parameters. The framework recovers optimal sample complexities for Gaussian mean and covariance estimation and provides a general blueprint for private estimation of unbounded high-dimensional exponential families, with potential broad impact on privacy-preserving data analysis in practice.

Abstract

We introduce a novel framework for differentially private (DP) statistical estimation via data truncation, addressing a key challenge in DP estimation when the data support is unbounded. Traditional approaches rely on problem-specific sensitivity analysis, limiting their applicability. By leveraging techniques from truncated statistics, we develop computationally efficient DP estimators for exponential family distributions, including Gaussian mean and covariance estimation, achieving near-optimal sample complexity. Previous works on exponential families only consider bounded or one-dimensional families. Our approach mitigates sensitivity through truncation while carefully correcting for the introduced bias using maximum likelihood estimation and DP stochastic gradient descent. Along the way, we establish improved uniform convergence guarantees for the log-likelihood function of exponential families, which may be of independent interest. Our results provide a general blueprint for DP algorithm design via truncated statistics.
Paper Structure (58 sections, 39 theorems, 47 equations, 4 algorithms)

This paper contains 58 sections, 39 theorems, 47 equations, 4 algorithms.

Table of Contents

  1. Introduction
  2. Data Truncation.
  3. Contributions
  4. Technical Contributions.
  5. Related Works
  6. DP Exponential Family Estimation.
  7. DP Gaussian Estimation.
  8. DP Empirical Risk Minimization.
  9. Truncated Statistics.
  10. Preliminaries
  11. Notation
  12. Neighboring Truncated Datasets
  13. Exponential Families
  14. Statistical Assumptions.
  15. Computational Subroutines.
  16. ...and 43 more sections

Key Result

Lemma 2.0

Fix $n\geq 1$ and let $N\in \mathbb{N}$. Let $D\in \mathbb{R}^{d\times N}$ be an $N$-sample dataset, $D'\in \mathbb{R}^{d\times N}$ be obtained from $D$ by modifying a single entry, and $S\subseteq \mathbb{R}^d$. Write $D_S, D_S'$ to denote the datasets obtained from $D, D'$ by discarding entries th

Theorems & Definitions (55)

  • Lemma 2.0
  • Theorem 3.1
  • Remark 3.2: Robustness against Existing Truncation
  • Theorem 3.4: Theorem II.1 and Theorem II.4 in bassily2014private
  • Lemma 3.4
  • Lemma 3.4
  • Lemma 3.4
  • Remark 3.5: High-Probability Guarantees; See \ref{['cor:exp-fam-estimation-high-prob']}
  • Theorem 4.1
  • Theorem 4.2
  • ...and 45 more