Membership Inference Attack Should Move On to Distributional Statistics for Distilled Generative Models

TL;DR

The paper identifies a privacy gap in distilled generative models where instance-level membership inference attacks (I-MIAs) fail to detect unauthorized data usage. It shows that distillation preserves a memory chain at the distribution level, such that student outputs align with the teacher's member distribution even when individual memorization signals vanish. Consequently, the authors propose distribution-based membership inference (D-MIA) and outline three guiding principles, introducing D-MIA as an exemplar framework based on deep-kernel MMD to audit distilled models. Empirical results demonstrate strong detection power of D-MIA across multiple datasets and distillation setups, highlighting the practical value of distributional auditing for upstream privacy and model provenance. The work argues for redefining privacy harm in model lifecycles and advocates continued development of distributional MIAs to address evolving generative-model deployments and governance needs.

Abstract

To detect unauthorized data usage in training large-scale generative models (e.g., ChatGPT or Midjourney), membership inference attacks (MIA) have proven effective in distinguishing a single training instance (a member) from a single non-training instance (a non-member). This success is mainly credited to a memorization effect: models tend to perform better on a member than a non-member. However, we find that standard MIAs fail against distilled generative models (i.e., student models) that are increasingly deployed in practice for efficiency (e.g., ChatGPT 4o-mini). Trained exclusively on data generated from a large-scale model (a teacher model), the student model lacks direct exposure to any members (teacher's training data), nullifying the memorization effect that standard MIAs rely on. This finding reveals a serious privacy loophole, where generation-service providers could deploy a student model whose teacher was potentially trained on unauthorized data, yet claim the deployed model is clean because it was not directly trained on such data. Hence, are distilled models inherently unauditable for upstream privacy violations, and should we discard them when we care about privacy? We contend no, as we uncover a memory chain connecting the student and teacher's member data: the distribution of student-generated data aligns more closely with the distribution of the teacher's members than with non-members, thus we can detect unauthorized data usage even when direct instance-level memorization is absent. This leads us to posit that MIAs on distilled generative models should shift from instance-level scores to distribution-level statistics. We further propose three principles of distribution-based MIAs for detecting unauthorized training data through distilled generative models, and validate our position through an exemplar framework. We lastly discuss the implications of our position.

Figures (5)

• Figure 1: Model distillation raises privacy concerns. MIA can detect unauthorized data in the teacher model but fails when only the student model is available. The reason behind this failure is that the student model is trained with the teacher’s outputs only, rather than the original data (i.e., members). More importantly, based on the failure of MIAs here, generation-service providers can only publish the student models as a service, to bypass unauthorized data detection and claim they do not use any unauthorized data for training.
• Figure 2: Model distillation for large-scale generative models. Diffusion models and LLMs rely on synthetic data produced by teacher models rather than the original training data to train student models. Thus, student models do not see the original training sets.
• Figure 3: Comparison of I-MIAs on teacher model EDM karras2022elucidating and student model DMD yin2024one using: (a) ReDiffuse li2024towards reveals membership signals in EDM via distinct reconstruction and re-noising losses between members and non-members on AFHQv2. (b) Applied to DMD, ReDiffuse fails to separate member from non-member instances. (c) Student outputs show stronger distributional alignment with member data when evaluated as instance sets via MMD gretton2012kernel.
• Figure 4: Overview of our two-phase MMD-based D-MIA framework, consisting of (1) deep-kernel MMD training phase (top left) and (2) detecting the Candidate Dataset phase (bottom left). We also propose a kernel ensemble strategy to improve detection robustness (right).
• Figure 5: Distribution analysis of D-MIA outputs across different member/non-member ratios within the candidate sets. Results are shown for distilled models against CIFAR10 (a, c) and FFHQ (b, d), where subfigures (a, b) report the results of DMD, while subfigures (c, d) present the results of Diff-Instruct.