Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lost in the Averages: A New Specific Setup to Evaluate Membership Inference Attacks Against Machine Learning Models

Nataša Krčo, Florent Guépin, Matthieu Meeus, Bogdan Kulynych, Yves-Alexandre de Montjoye

TL;DR

This work proposes a novel use of the leave-one-out game, used in existing work exclusively to audit differential privacy guarantees, and formalizes it and shows that it provides an accurate estimate of the privacy risk posed by a given adversary for a record in its specific dataset.

Abstract

Synthetic data generators and machine learning models can memorize their training data, posing privacy concerns. Membership inference attacks (MIAs) are a standard method of estimating the privacy risk of these systems. The risk of individual records is typically computed by evaluating MIAs in a record-specific privacy game. We analyze the record-specific privacy game commonly used for evaluating attackers under realistic assumptions (the \textit{traditional} game) -- particularly for synthetic tabular data -- and show that it averages a record's privacy risk across datasets. We show this implicitly assumes the dataset a record is part of has no impact on the record's risk, providing a misleading risk estimate when a specific model or synthetic dataset is released. Instead, we propose a novel use of the leave-one-out game, used in existing work exclusively to audit differential privacy guarantees, and call this the \textit{model-seeded} game. We formalize it and show that it provides an accurate estimate of the privacy risk posed by a given adversary for a record in its specific dataset. We instantiate and evaluate the state-of-the-art MIA for synthetic data generators in the traditional and model-seeded privacy games, and show across multiple datasets and models that the two privacy games indeed result in different risk scores, with up to 94\% of high-risk records being overlooked by the traditional game. We further show that records in smaller datasets and models not protected by strong differential privacy guarantees tend to have a larger gap between risk estimates. Taken together, our results show that the model-seeded setup yields a risk estimate specific to a certain model or synthetic dataset released and in line with the standard notion of privacy leakage from prior work, meaningfully different from the dataset-averaged risk provided by the traditional privacy game.

Lost in the Averages: A New Specific Setup to Evaluate Membership Inference Attacks Against Machine Learning Models

TL;DR

This work proposes a novel use of the leave-one-out game, used in existing work exclusively to audit differential privacy guarantees, and formalizes it and shows that it provides an accurate estimate of the privacy risk posed by a given adversary for a record in its specific dataset.

Abstract

Synthetic data generators and machine learning models can memorize their training data, posing privacy concerns. Membership inference attacks (MIAs) are a standard method of estimating the privacy risk of these systems. The risk of individual records is typically computed by evaluating MIAs in a record-specific privacy game. We analyze the record-specific privacy game commonly used for evaluating attackers under realistic assumptions (the \textit{traditional} game) -- particularly for synthetic tabular data -- and show that it averages a record's privacy risk across datasets. We show this implicitly assumes the dataset a record is part of has no impact on the record's risk, providing a misleading risk estimate when a specific model or synthetic dataset is released. Instead, we propose a novel use of the leave-one-out game, used in existing work exclusively to audit differential privacy guarantees, and call this the \textit{model-seeded} game. We formalize it and show that it provides an accurate estimate of the privacy risk posed by a given adversary for a record in its specific dataset. We instantiate and evaluate the state-of-the-art MIA for synthetic data generators in the traditional and model-seeded privacy games, and show across multiple datasets and models that the two privacy games indeed result in different risk scores, with up to 94\% of high-risk records being overlooked by the traditional game. We further show that records in smaller datasets and models not protected by strong differential privacy guarantees tend to have a larger gap between risk estimates. Taken together, our results show that the model-seeded setup yields a risk estimate specific to a certain model or synthetic dataset released and in line with the standard notion of privacy leakage from prior work, meaningfully different from the dataset-averaged risk provided by the traditional privacy game.
Paper Structure (37 sections, 3 theorems, 14 equations, 6 figures, 2 tables)

This paper contains 37 sections, 3 theorems, 14 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Background
  4. Synthetic data generation
  5. MIA development
  6. Threat model.
  7. Shadow modeling.
  8. Computing a membership score.
  9. Differential privacy and its hypothesis-testing interpretation
  10. Record-specific MIA evaluation
  11. Traditional privacy game
  12. Model-seeded privacy game
  13. The relationship between the games and privacy risk
  14. Experimental setup
  15. Data, target model and attack details
  16. ...and 22 more sections

Key Result

Proposition 1

For any fixed target record $x$, partial dataset $\bar{D} \in \mathbb{D}^{n-1}$, training algorithm $T(\cdot)$, and attack $\phi(\cdot)$, we have w.p. $1 - \rho$ for $\rho \in (0, 1)$ over $N$ random coin flips, i.e., fresh seed draws, in the model-seeded game:

Figures (6)

  • Figure 1: Risk for all 1000 records in $D$ sampled from the Adult dataset (Synthpop). (a) per-record model-seeded and traditional risks. The shaded area marks all the high-risk records missed in the traditional setup for high-risk threshold $t=0.8$. (b) histogram of per-record absolute differences between the model-seeded and traditional risks.
  • Figure 2: (a) Miss rate for different high-risk thresholds $t$ for SDG setups. Note that for Census and Baynet, there are no records with $R^\text{MS}>0.9$, therefore the miss rate is not defined. (b) CDFs of $R^\text{MS}$ and $R^\text{T}$ across records in target dataset $D$ drawn from Adult dataset with $\theta$ Synthpop.
  • Figure 3: (a) RMSD between model-seeded and traditional risk per target dataset size. (b) Model-seeded and traditional risk values per target dataset size. For both figures, values are computed across $20$ target records.
  • Figure 4: Model-seeded and traditional risk for $20$ records for Adult dataset and $\theta$ Privbayes with varying $\varepsilon$.
  • Figure 5: Model-seeded risks of one target record within 15 different datasets and its traditional risk.
  • ...and 1 more figures

Theorems & Definitions (7)

  • Definition 1
  • Definition 2: Traditional record-specific privacy game
  • Definition 3: Model-seeded record-specific privacy game
  • Proposition 1: Model-seeded game converges to DPD risk
  • Proposition 2: Traditional game converges to average privacy risk
  • Proposition 2: Model-seeded game converges to DPD risk
  • proof : Proof