Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differentially Private Learning of Exponential Distributions: Adaptive Algorithms and Tight Bounds

Bar Mahpud, Or Sheffet

TL;DR

This work studies the problem of learning the rate parameter $\lambda$ of the exponential distribution under differential privacy, aiming for a learned distribution $\mathrm{Exp}(\tilde{\lambda})$ that is close to the truth in total variation. It introduces two complementary pure-DP learners—a private MLE with clipping and Laplace noise, and a quantile-based estimator that exploits the fact that the $(1-1/e)$-quantile equals $1/\lambda$—and combines them into an adaptive best-of-both algorithm that achieves near-optimal sample complexity across regimes of $\lambda$. The results extend to Pareto distributions via logarithmic reduction, and establish nearly matching lower bounds via group privacy; they also show how approximate DP can remove the need for externally supplied $\lambda$-bounds. Together, these findings provide the first tight DP characterization for learning exponential (and via reduction, Pareto) distributions and demonstrate the power of adaptive strategies for heavy-tailed statistics in private settings.

Abstract

We study the problem of learning exponential distributions under differential privacy. Given $n$ i.i.d.\ samples from $\mathrm{Exp}(λ)$, the goal is to privately estimate $λ$ so that the learned distribution is close in total variation distance to the truth. We present two complementary pure DP algorithms: one adapts the classical maximum likelihood estimator via clipping and Laplace noise, while the other leverages the fact that the $(1-1/e)$-quantile equals $1/λ$. Each method excels in a different regime, and we combine them into an adaptive best-of-both algorithm achieving near-optimal sample complexity for all $λ$. We further extend our approach to Pareto distributions via a logarithmic reduction, prove nearly matching lower bounds using packing and group privacy \cite{Karwa2017FiniteSD}, and show how approximate $(ε,δ)$-DP removes the need for externally supplied bounds. Together, these results give the first tight characterization of exponential distribution learning under DP and illustrate the power of adaptive strategies for heavy-tailed laws.

Differentially Private Learning of Exponential Distributions: Adaptive Algorithms and Tight Bounds

TL;DR

This work studies the problem of learning the rate parameter of the exponential distribution under differential privacy, aiming for a learned distribution that is close to the truth in total variation. It introduces two complementary pure-DP learners—a private MLE with clipping and Laplace noise, and a quantile-based estimator that exploits the fact that the -quantile equals —and combines them into an adaptive best-of-both algorithm that achieves near-optimal sample complexity across regimes of . The results extend to Pareto distributions via logarithmic reduction, and establish nearly matching lower bounds via group privacy; they also show how approximate DP can remove the need for externally supplied -bounds. Together, these findings provide the first tight DP characterization for learning exponential (and via reduction, Pareto) distributions and demonstrate the power of adaptive strategies for heavy-tailed statistics in private settings.

Abstract

We study the problem of learning exponential distributions under differential privacy. Given i.i.d.\ samples from , the goal is to privately estimate so that the learned distribution is close in total variation distance to the truth. We present two complementary pure DP algorithms: one adapts the classical maximum likelihood estimator via clipping and Laplace noise, while the other leverages the fact that the -quantile equals . Each method excels in a different regime, and we combine them into an adaptive best-of-both algorithm achieving near-optimal sample complexity for all . We further extend our approach to Pareto distributions via a logarithmic reduction, prove nearly matching lower bounds using packing and group privacy \cite{Karwa2017FiniteSD}, and show how approximate -DP removes the need for externally supplied bounds. Together, these results give the first tight characterization of exponential distribution learning under DP and illustrate the power of adaptive strategies for heavy-tailed laws.

Paper Structure

This paper contains 19 sections, 40 theorems, 115 equations, 7 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Differential Privacy
  4. Probabilistic Background
  5. Exponential Distribution
  6. Pareto Distribution
  7. Pure Differentially Private Algorithms for Exponential Distribution Learning
  8. Learning $\lambda$ from the MLE
  9. Learning $\lambda$ from the $(1-1/e)$-Quantile
  10. Best of Both Algorithms
  11. Applications to Pareto Distributions
  12. Lower Bound for Exponential Distribution Learning under Pure Differential Privacy
  13. Finding Initial Bounds Using Approximate Differential Privacy
  14. Conclusion
  15. Missing Proofs
  16. ...and 4 more sections

Key Result

Lemma 2.3

If $\mathcal{M}_1$ is $(\epsilon_1,\delta_1)$-DP and $\mathcal{M}_2$ is $(\epsilon_2,\delta_2)$-DP, then their sequential composition $(\mathcal{M}_1,\mathcal{M}_2)$ is $(\epsilon_1+\epsilon_2,\ \delta_1+\delta_2)$-DP. In particular, composing $k$ pure $\epsilon$-DP mechanisms yields $k\epsilon$-DP.

Theorems & Definitions (71)

  • Definition 2.1: Differential Privacy
  • Definition 2.2: Laplace Mechanism
  • Lemma 2.3: Basic Composition
  • Definition 2.4: Total Variation Distance
  • Lemma 2.5: Dvoretzky–Kiefer–Wolfowitz (DKW) Inequality
  • Lemma 2.6: Multiplicative Chernoff Bound
  • Definition 2.7: Maximum Likelihood Estimator for the Exponential Distribution
  • Corollary 2.9
  • Remark
  • Lemma 3.1: Privacy of \ref{['alg:quantile']}
  • ...and 61 more