Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspective

TL;DR

This work reframes synthetic data anonymization through a model-centric lens aligned with GDPR concepts, arguing that identifiability risks depend on the underlying generative model and its outputs rather than solely on dataset-level anonymization. It maps GDPR risks—singling out, linkability, and inferences—to concrete privacy attacks (differencing, membership inference, attribute inference, and reconstruction) and adopts a motivated intruder framework to evaluate real-world threat scenarios. The authors compare Differential Privacy (DP) and Similarity-based Privacy Metrics (SBPMs), showing that DP can provide robust anonymization under appropriate conditions, while SBPMs typically fail to offer adequate protection. The findings advocate for regulator and practitioner adoption of a model-centric privacy assessment, guided by principled DP implementations, to achieve trustworthy synthetic data sharing. This approach bridges regulatory notions and privacy attacks, informing governance, policy, and engineering choices in synthetic data ecosystems.

Abstract

Training generative machine learning models to produce synthetic tabular data has become a popular approach for enhancing privacy in data sharing. As this typically involves processing sensitive personal information, releasing either the trained model or generated synthetic datasets can still pose privacy risks. Yet, recent research, commercial deployments, and privacy regulations like the General Data Protection Regulation (GDPR) largely assess anonymity at the level of an individual dataset. In this paper, we rethink anonymity claims about synthetic data from a model-centric perspective and argue that meaningful assessments must account for the capabilities and properties of the underlying generative model and be grounded in state-of-the-art privacy attacks. This perspective better reflects real-world products and deployments, where trained models are often readily accessible for interaction or querying. We interpret the GDPR's definitions of personal data and anonymization under such access assumptions to identify the types of identifiability risks that must be mitigated and map them to privacy attacks across different threat settings. We then argue that synthetic data techniques alone do not ensure sufficient anonymization. Finally, we compare the two mechanisms most commonly used alongside synthetic data -- Differential Privacy (DP) and Similarity-based Privacy Metrics (SBPMs) -- and argue that while DP can offer robust protections against identifiability risks, SBPMs lack adequate safeguards. Overall, our work connects regulatory notions of identifiability with model-centric privacy attacks, enabling more responsible and trustworthy regulatory assessment of synthetic data systems by researchers, practitioners, and policymakers.

Figures (5)

• Figure 1: Synthetic data generation using generative models.
• Figure 2: Privacy attacks on synthetic data: an adversary with black or white-box access to the model (and side information) extracts private information from the train data.
• Figure 3: Synthetic data generation with Differential Privacy (DP); the model is trained while satisfying DP.
• Figure 4: Synthetic data generation with Similarity-based Privacy Metrics (SBPMs); the privacy of synthetic data is measured through SBPMs.
• Figure 5: Base-case examples of synthetic data with privacy guaranteed by DP and SBPMs.