On Membership Inference Attacks in Knowledge Distillation
Ziyao Cui, Minxing Zhang, Jian Pei
TL;DR
This work interrogates the assumption that knowledge distillation inherently improves privacy by studying membership inference attacks (MIAs) on six teacher–student LLM pairs across six MIAs. It reveals that distillation does not consistently reduce aggregate MIA accuracy and can even raise member-specific leakage due to mixed supervision signals that reinforce memorization on vulnerable data. The authors diagnose the cause as misalignment between ground-truth supervision and teacher predictions, and propose three defenses: distillation restricted to non-vulnerable data, Bottleneck Projection to constrain representations, and NoNorm to replace layer normalization. Empirical results show these methods reduce both aggregate and member-specific MIA success while preserving utility, offering practical privacy-utility improvements for distilled LLMs. The work highlights the need for privacy-aware distillation strategies and points to future directions including theoretical analyses and integration with formal privacy mechanisms.
Abstract
Large language models (LLMs) are trained on massive corpora that may contain sensitive information, creating privacy risks under membership inference attacks (MIAs). Knowledge distillation is widely used to compress LLMs into smaller student models, but its privacy implications are poorly understood. We systematically evaluate how distillation affects MIA vulnerability across six teacher-student model pairs and six attack methods. We find that distilled student models do not consistently exhibit lower MIA success than their teacher models, and in some cases demonstrate substantially higher member-specific attack success, challenging the assumption that knowledge distillation inherently improves privacy. We attribute this to mixed supervision in distillation: for vulnerable training data points, teacher predictions often align with ground-truth labels, causing student models to learn overly confident predictions that amplify the separability between members and non-members; conversely, for non-vulnerable points, teacher predictions and ground truth frequently diverge, providing inconsistent learning signals. To mitigate this, we propose three practical interventions -- restricting distillation to non-vulnerable points, adding a low-dimensional Bottleneck Projection, and a normalization variant (NoNorm). Experiments show these methods reduce both aggregate and member-specific MIA success while preserving model utility, improving privacy-utility trade-offs for distilled LLMs.