Quantifying Membership Disclosure Risk for Tabular Synthetic Data Using Kernel Density Estimators

TL;DR

This work proposes a practical and effective method to quantify membership disclosure risk in tabular synthetic datasets using kernel density estimators (KDEs) and proposes two attack models: a'True Distribution Attack', which assumes privileged access to training data, and a more realistic, implementable'Realistic Attack', which uses auxiliary data without true membership labels.

Abstract

The use of synthetic data has become increasingly popular as a privacy-preserving alternative to sharing real datasets, especially in sensitive domains such as healthcare, finance, and demography. However, the privacy assurances of synthetic data are not absolute, and remain susceptible to membership inference attacks (MIAs), where adversaries aim to determine whether a specific individual was present in the dataset used to train the generator. In this work, we propose a practical and effective method to quantify membership disclosure risk in tabular synthetic datasets using kernel density estimators (KDEs). Our KDE-based approach models the distribution of nearest-neighbour distances between synthetic data and the training records, allowing probabilistic inference of membership and enabling robust evaluation via ROC curves. We propose two attack models: a 'True Distribution Attack', which assumes privileged access to training data, and a more realistic, implementable 'Realistic Attack' that uses auxiliary data without true membership labels. Empirical evaluations across four real-world datasets and six synthetic data generators demonstrate that our method consistently achieves higher F1 scores and sharper risk characterization than a prior baseline approach, without requiring computationally expensive shadow models. The proposed method provides a practical framework and metric for quantifying membership disclosure risk in synthetic data, which enables data custodians to conduct a post-generation risk assessment prior to releasing their synthetic datasets for downstream use. The datasets and codes for this study are available at https://github.com/PyCoder913/MIA-KDE.

Figures (17)

• Figure 1: A visual depiction of the process flow of the True Distribution Attack: First, the attack dataset is constructed using training and unseen records; For each record in the attack dataset, its nearest neighbour distance from the synthetic data is obtained; Separate KDEs are fitted to the member distances and non-member distances.
• Figure 2: A visual depiction of the process flow of the Realistic Attack: The attack dataset is constructed first. As the data holder, we know the true membership labels, but the adversary does not; For each record in the attack dataset, its nearest neighbour distance $d$ from the synthetic data is obtained. A distance threshold $\tau$ is set. A record is labelled as 'supposed member' if the distance $d < \tau$, otherwise 'supposed non-member'; Separate KDEs are fitted to the 'supposed member distances' and 'supposed non-member distances.'
• Figure 3: F1 scores (MIA risk) for different synthetic datasets across all four real datasets: True Distribution Attack vs. Realistic vs. Method 1.
• Figure 4: True distribution attack accuracies (left) and F1 scores (right) on the MIMIC-IV synthetic datasets.
• Figure 5: MIMIC-IV data: Distribution of nearest-neighbour distances between the training records (in $D_{\text{attack}}$) and various synthetic datasets. The member and non-member distances are modelled separately for each synthetic data.

Theorems & Definitions (2)

• Proposition 3.1
• proof