Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gaussian Differential Private Bootstrap by Subsampling

Holger Dette, Carina Graw

TL;DR

This work tackles uncertainty quantification under differential privacy for large datasets by introducing a private empirical $m$ out of $n$ bootstrap under Gaussian Differential Privacy. By subsampling to size $m$ and privatizing bootstrap estimators with Gaussian noise, the method reduces privacy-induced variance and computational cost while preserving asymptotic privacy and consistency; the authors derive rigorous privacy guarantees and consistency results, and offer practical guidance on choosing $m$ and $B$. Through simulations on a truncated-normal mean and a regularized logistic regression problem, the approach yields valid confidence intervals with shorter lengths and better finite-sample performance than existing private bootstrap methods, with substantial computational savings. The results demonstrate the method’s practicality for large-scale private inference, including applications to mean estimation and regression under GDP and provide a framework for comparing private bootstrap approaches in terms of accuracy, coverage, and efficiency.

Abstract

Bootstrap is a common tool for quantifying uncertainty in data analysis. However, besides additional computational costs in the application of the bootstrap on massive data, a challenging problem in bootstrap based inference under Differential Privacy consists in the fact that it requires repeated access to the data. As a consequence, bootstrap based differentially private inference requires a significant increase of the privacy budget, which on the other hand comes with a substantial loss in statistical accuracy. A potential solution to reconcile the conflicting goals of statistical accuracy and privacy is to analyze the data under parametric model assumptions and in the last decade, several parametric bootstrap methods for inference under privacy have been investigated. However, uncertainty quantification by parametric bootstrap is only valid if the the quantities of interest can be identified as the parameters of a statistical model and the imposed model assumptions are (at least approximately) satisfied. An alternative to parametric methods is the empirical bootstrap that is a widely used tool for non-parametric inference and well studied in the non-private regime. However, under privacy, less insight is available. In this paper, we propose a private empirical $m$ out of $n$ bootstrap and validate its consistency and privacy guarantees under Gaussian Differential Privacy. Compared to the the private $n$ out of $n$ bootstrap, our approach has several advantages. First, it comes with less computational costs, in particular for massive data. Second, the proposed procedure needs less additional noise in the bootstrap iterations, which leads to an improved statistical accuracy while asymptotically guaranteeing the same level of privacy. Third, we demonstrate much better finite sample properties compared to the currently available procedures.

Gaussian Differential Private Bootstrap by Subsampling

TL;DR

This work tackles uncertainty quantification under differential privacy for large datasets by introducing a private empirical out of bootstrap under Gaussian Differential Privacy. By subsampling to size and privatizing bootstrap estimators with Gaussian noise, the method reduces privacy-induced variance and computational cost while preserving asymptotic privacy and consistency; the authors derive rigorous privacy guarantees and consistency results, and offer practical guidance on choosing and . Through simulations on a truncated-normal mean and a regularized logistic regression problem, the approach yields valid confidence intervals with shorter lengths and better finite-sample performance than existing private bootstrap methods, with substantial computational savings. The results demonstrate the method’s practicality for large-scale private inference, including applications to mean estimation and regression under GDP and provide a framework for comparing private bootstrap approaches in terms of accuracy, coverage, and efficiency.

Abstract

Bootstrap is a common tool for quantifying uncertainty in data analysis. However, besides additional computational costs in the application of the bootstrap on massive data, a challenging problem in bootstrap based inference under Differential Privacy consists in the fact that it requires repeated access to the data. As a consequence, bootstrap based differentially private inference requires a significant increase of the privacy budget, which on the other hand comes with a substantial loss in statistical accuracy. A potential solution to reconcile the conflicting goals of statistical accuracy and privacy is to analyze the data under parametric model assumptions and in the last decade, several parametric bootstrap methods for inference under privacy have been investigated. However, uncertainty quantification by parametric bootstrap is only valid if the the quantities of interest can be identified as the parameters of a statistical model and the imposed model assumptions are (at least approximately) satisfied. An alternative to parametric methods is the empirical bootstrap that is a widely used tool for non-parametric inference and well studied in the non-private regime. However, under privacy, less insight is available. In this paper, we propose a private empirical out of bootstrap and validate its consistency and privacy guarantees under Gaussian Differential Privacy. Compared to the the private out of bootstrap, our approach has several advantages. First, it comes with less computational costs, in particular for massive data. Second, the proposed procedure needs less additional noise in the bootstrap iterations, which leads to an improved statistical accuracy while asymptotically guaranteeing the same level of privacy. Third, we demonstrate much better finite sample properties compared to the currently available procedures.
Paper Structure (17 sections, 8 theorems, 63 equations, 7 tables, 4 algorithms)

This paper contains 17 sections, 8 theorems, 63 equations, 7 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. $f$-Differential Privacy
  4. $f$-DP Bootstrap
  5. $m$ out of $n$ Bootstrap
  6. Privacy Guarantee.
  7. Consistency.
  8. Choice of $m$ and $B$.
  9. Computational time.
  10. Simulations
  11. Mean of a truncated normal distribution
  12. Regularized logistic regression
  13. Additional Simulation results
  14. Proofs
  15. Proof of \ref{['thm:asymp_privacy_bootstrap_mn']}
  16. ...and 2 more sections

Key Result

Theorem 2.1

Let $\theta(S_n)$ be a univariate statistic computed on a dataset $S_n$ of size $n$ and define the sensitivity of $\theta$ as Let $\xi \sim N(0, \sigma_\mu^2)$, where $\sigma_\mu^2=\Delta^2 / \mu^2$, then the Gaussian mechanism is $\mu$-GDP.

Theorems & Definitions (14)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3: $\mu$-Gaussian Differential Privacy
  • Theorem 2.1: Gaussian Mechanism
  • Theorem 2.2
  • Lemma 2.1
  • Theorem 3.1
  • Remark 3.1
  • Theorem 3.2
  • Corollary 3.1
  • ...and 4 more