Epsilon*: Privacy Metric for Machine Learning Models

TL;DR

Epsilon* introduces an empirical, model-instance privacy metric that quantifies privacy risk via a hypothesis-test framework for membership inference attacks while requiring only black-box access to predictions. It provides a formal lower bound on the model-level DP parameter $\epsilon$ within the ($\epsilon$, $\delta$)-privacy framework and clarifies connections to mechanism-level bounds through $\bar{\epsilon}$. A parametric variant improves numerical stability by fitting Gaussian losses and using a transformed loss space to compute Epsilon* more robustly. Across two public datasets, the approach demonstrates that DP training can substantially reduce Epsilon* (up to 800% relative reduction) and enables visualization of privacy-utility trade-offs, supporting independent privacy auditing and informed deployment decisions.

Abstract

We introduce Epsilon*, a new privacy metric for measuring the privacy risk of a single model instance prior to, during, or after deployment of privacy mitigation strategies. The metric requires only black-box access to model predictions, does not require training data re-sampling or model re-training, and can be used to measure the privacy risk of models not trained with differential privacy. Epsilon* is a function of true positive and false positive rates in a hypothesis test used by an adversary in a membership inference attack. We distinguish between quantifying the privacy loss of a trained model instance, which we refer to as empirical privacy, and quantifying the privacy loss of the training mechanism which produces this model instance. Existing approaches in the privacy auditing literature provide lower bounds for the latter, while our metric provides an empirical lower bound for the former by relying on an ($ε$, $δ$)-type of quantification of the privacy of the trained model instance. We establish a relationship between these lower bounds and show how to implement Epsilon* to avoid numerical and noise amplification instability. We further show in experiments on benchmark public data sets that Epsilon* is sensitive to privacy risk mitigation by training with differential privacy (DP), where the value of Epsilon* is reduced by up to 800% compared to the Epsilon* values of non-DP trained baseline models. This metric allows privacy auditors to be independent of model owners, and enables visualizing the privacy-utility landscape to make informed decisions regarding the trade-offs between model privacy and utility.

Figures (6)

• Figure 1: Privacy-Utility trade-off for the Purchase-100 data set. Each point correspons to one strategy (same hyperparameter set and privacy mitigation), averaged over five model instances. Error bars correspond to the minimum and maximum values over the five model instances.
• Figure 2: Privacy-Utility trade-off for the Adult data set. Each point correspons to one strategy (same hyperparameter set and privacy mitigation), averaged over five model instances. Error bars correspond to the minimum and maximum values over the five model instances.
• Figure 3: First two expressions in (\ref{['eps_star_levels']}), when evaluating the true versus the empirical CDFs from sampled data
• Figure 4: Epsilon* from empirical CDFs vs parametric fitted distributions to loss data when the true training and non-training distributions are identical
• Figure 5: Epsilon* from empirical CDFs vs parametric fitted distributions to loss data when the true training and non-training distributions are $\Gamma(k_1, \theta_1)$, and $\Gamma(k_1 + d, \theta_1)$ respectively, for $d \in \{0,1,2,3\}$, $k_1 = 2$, $\theta_1$ = 5.

Theorems & Definitions (8)

• Theorem 1: kairouz2015composition
• Definition 1: Empirical privacy
• Definition 2: Epsilon*
• Theorem 2
• Theorem 3
• Theorem 4
• proof
• proof : Proof of Theorem \ref{['eps_star_vs_mech']}