Improved subsample-and-aggregate via the private modified winsorized mean

TL;DR

This paper addresses the challenge of privately estimating a univariate mean to be used as the aggregator in subsample-and-aggregate (SSA) while tolerating data contamination and weak input-bounds assumptions. It introduces the private modified winsorized mean (PMW mean), a univariate differentially private estimator that blends Lugosi's modified winsorized mean with private quantile estimation, achieving minimax-optimal performance across broad distribution classes, including adversarial contamination, and providing a finite-sample SSA deviation bound. A practical variant (\tilde{\mu}'_p) is proposed for improved applicability, along with a zero-concentrated DP quantile approach and a correction to a prior result. Empirical studies across simulations and a real healthcare dataset demonstrate that PMW-based SSA often outperforms existing private aggregators, with robustness to contamination and favorable behavior in small samples, thereby offering a scalable, privacy-preserving route for private mean aggregation in diverse analyses.

Abstract

We develop a univariate, differentially private mean estimator, called the private modified winsorized mean, designed to be used as the aggregator in subsample-and-aggregate. We demonstrate, via real data analysis, that common differentially private multivariate mean estimators may not perform well as the aggregator, even in large datasets, motivating our developments.We show that the modified winsorized mean is minimax optimal for several, large classes of distributions, even under adversarial contamination. We also demonstrate that, empirically, the private modified winsorized mean performs well compared to other private mean estimates. We consider the modified winsorized mean as the aggregator in subsample-and-aggregate, deriving a finite sample deviations bound for a subsample-and-aggregate estimate generated with the new aggregator. This result yields two important insights: (i) the optimal choice of subsamples depends on the bias of the estimator computed on the subsamples, and (ii) the rate of convergence of the subsample-and-aggregate estimator depends on the robustness of the estimator computed on the subsamples.

Figures (6)

• Figure 1: The empirical $\log MSE$ of $\tilde{\mu}_p,\ \tilde{\mu}$, with $\eta=0$ and $\eta=0.3$, compared to the estimator of Bun2019, Bun, with $t=0.1$, compared across a variety of different population distributions (Gaussian, mixture of Gaussians, Skewed, Heavy-Tailed, and a Contaminated Gaussian) and different privacy budgets ($\rho$).
• Figure 2: Actual mean values, non-private estimated mean values and private estimated mean values of K10 scores of healthcare workers throughout the COVID-19 pandemic.
• Figure 3: The empirical $\log MSE$ of all estimators, compared across a variety of different population distributions (Mixture of Gaussians, Skewed, Heavy-Tailed, and a Contaminated Gaussian) at $\rho=0.1$.
• Figure 4: The empirical $\log MSE$ of all estimators, compared across a variety of different population distributions (Mixture of Gaussians, Skewed, Heavy-Tailed, and a Contaminated Gaussian) at $\rho=0.5$.
• Figure 5: The empirical $\log MSE$ of all estimators, compared across a variety of different population distributions (Mixture of Gaussians, Skewed, Heavy-Tailed, and a Contaminated Gaussian) at $\rho=1$.

Theorems & Definitions (21)

• Definition 1
• Proposition 1
• Proposition 2
• Definition 2
• Theorem 1
• Remark 1: On choosing $\beta$
• Remark 2: On choosing $u$ and $\ell$
• Remark 3: Weakening Condition \ref{['cond::bounds']}
• Remark 4: Subgaussian distributions and more
• Remark 5: Multivariate case