Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Private Minimum Hellinger Distance Estimation via Hellinger Distance Differential Privacy

Fengnan Deng, Anand N. Vidyashankar

TL;DR

This work develops private minimum Hellinger distance estimators (PMHDE) that achieve robustness and efficiency under privacy constraints by embedding Hellinger differential privacy (HDP) into null-robust minimum-divergence estimation. By calibrating Gaussian and Laplace noise per iteration and using symmetric matrix mechanisms, the authors design private gradient descent and Newton-Raphson algorithms that preserve the asymptotic efficiency of the non-private MHDE. They establish sharp gradient/Hessian sensitivity bounds, prove HDP-compatible composition and group privacy properties, and show that PMHDE shares the same asymptotic distribution as MHDE under regularity. The framework also provides private confidence intervals and demonstrates strong finite-sample performance and robustness to contamination via numerical experiments. Overall, the paper delivers a principled, practically applicable approach to private, robust, and efficient density-based parameter estimation using HDP within PDP.

Abstract

Objective functions based on Hellinger distance yield robust and efficient estimators of model parameters. Motivated by privacy and regulatory requirements encountered in contemporary applications, we derive in this paper \emph{private minimum Hellinger distance estimators}. The estimators satisfy a new privacy constraint, namely, Hellinger differential privacy, while retaining the robustness and efficiency properties. We demonstrate that Hellinger differential privacy shares several features of standard differential privacy while allowing for sharper inference. Additionally, for computational purposes, we also develop Hellinger differentially private gradient descent and Newton-Raphson algorithms. We illustrate the behavior of our estimators in finite samples using numerical experiments and verify that they retain robustness properties under gross-error contamination.

Private Minimum Hellinger Distance Estimation via Hellinger Distance Differential Privacy

TL;DR

This work develops private minimum Hellinger distance estimators (PMHDE) that achieve robustness and efficiency under privacy constraints by embedding Hellinger differential privacy (HDP) into null-robust minimum-divergence estimation. By calibrating Gaussian and Laplace noise per iteration and using symmetric matrix mechanisms, the authors design private gradient descent and Newton-Raphson algorithms that preserve the asymptotic efficiency of the non-private MHDE. They establish sharp gradient/Hessian sensitivity bounds, prove HDP-compatible composition and group privacy properties, and show that PMHDE shares the same asymptotic distribution as MHDE under regularity. The framework also provides private confidence intervals and demonstrates strong finite-sample performance and robustness to contamination via numerical experiments. Overall, the paper delivers a principled, practically applicable approach to private, robust, and efficient density-based parameter estimation using HDP within PDP.

Abstract

Objective functions based on Hellinger distance yield robust and efficient estimators of model parameters. Motivated by privacy and regulatory requirements encountered in contemporary applications, we derive in this paper \emph{private minimum Hellinger distance estimators}. The estimators satisfy a new privacy constraint, namely, Hellinger differential privacy, while retaining the robustness and efficiency properties. We demonstrate that Hellinger differential privacy shares several features of standard differential privacy while allowing for sharper inference. Additionally, for computational purposes, we also develop Hellinger differentially private gradient descent and Newton-Raphson algorithms. We illustrate the behavior of our estimators in finite samples using numerical experiments and verify that they retain robustness properties under gross-error contamination.
Paper Structure (40 sections, 30 theorems, 243 equations, 6 figures, 22 tables, 2 algorithms)

This paper contains 40 sections, 30 theorems, 243 equations, 6 figures, 22 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background, notations, and Hellinger distance differential privacy
  3. Private minimum Hellinger distance estimation
  4. Minimum Hellinger distance estimation
  5. Almost sure local convexity
  6. Private optimization
  7. Utility of PMHDE
  8. Efficiency for private MHDEs
  9. Private confidence interval
  10. Numerical experiments
  11. Extensions and concluding remarks
  12. Proof
  13. Proof of Theorem \ref{['thm:PDP']} and \ref{['thm:HDP']}
  14. Proof of Theorem \ref{['thm:PDP_mech']} and Proposition \ref{['prop:HDP_mech']}
  15. Proof of Corollary \ref{['cor:composition']}
  16. ...and 25 more sections

Key Result

Theorem 2.1

Figures (6)

  • Figure 1: Private gradient descent path
  • Figure 2: Private gradient descent trajectory
  • Figure 3: Private Newton's method path
  • Figure 4: Private Newton's method trajectory
  • Figure 5: Private and non-private gradient descent 95% confidence interval coverage
  • ...and 1 more figures

Theorems & Definitions (39)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Definition 2.4
  • Remark 2.1
  • Theorem 2.1
  • Theorem 2.2
  • Theorem 2.3
  • Corollary 2.1
  • Proposition 2.1
  • ...and 29 more