Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models

TL;DR

This work investigates privacy risks for pre-trained large language models under the label-only membership inference setting, where only generated tokens are observable. It shows that existing label-only attacks relying on robustness signals are ineffective against pre-trained LLMs due to strong generalization and coarse perturbations. Introducing PETAL, a label-only MIA based on per-token semantic similarity to approximate output probabilities and a derived perplexity, the authors demonstrate performance on par with logits-based attacks across WikiMIA and MIMIR benchmarks and five open-source LLMs. PETAL remains effective against some defenses and extends to closed-source models, highlighting practical privacy risks in real-world LLM deployments and motivating the need for robust defensive strategies during pre-training and data curation.

Abstract

Membership Inference Attacks (MIAs) aim to predict whether a data sample belongs to the model's training set or not. Although prior research has extensively explored MIAs in Large Language Models (LLMs), they typically require accessing to complete output logits (\ie, \textit{logits-based attacks}), which are usually not available in practice. In this paper, we study the vulnerability of pre-trained LLMs to MIAs in the \textit{label-only setting}, where the adversary can only access generated tokens (text). We first reveal that existing label-only MIAs have minor effects in attacking pre-trained LLMs, although they are highly effective in inferring fine-tuning datasets used for personalized LLMs. We find that their failure stems from two main reasons, including better generalization and overly coarse perturbation. Specifically, due to the extensive pre-training corpora and exposing each sample only a few times, LLMs exhibit minimal robustness differences between members and non-members. This makes token-level perturbations too coarse to capture such differences. To alleviate these problems, we propose \textbf{PETAL}: a label-only membership inference attack based on \textbf{PE}r-\textbf{T}oken sem\textbf{A}ntic simi\textbf{L}arity. Specifically, PETAL leverages token-level semantic similarity to approximate output probabilities and subsequently calculate the perplexity. It finally exposes membership based on the common assumption that members are `better' memorized and have smaller perplexity. We conduct extensive experiments on the WikiMIA benchmark and the more challenging MIMIR benchmark. Empirically, our PETAL performs better than the extensions of existing label-only attacks against personalized LLMs and even on par with other advanced logit-based attacks across all metrics on five prevalent open-source LLMs.

Figures (8)

• Figure 1: General attack pipeline of our PETAL.
• Figure 2: The similarity scores distribution of members and non-members. There is no notable gap in the scores between members and non-members.
• Figure 3: The distribution of semantic similarity and probability of tokens from two randomly sampled text.
• Figure 4: AUC values of our PETAL against Pythia-6.9B. The granularity of probability estimation is set as 1, 2, 4, 8, and 16. To avoid overcrowding the figure, the attack results on other subsets of MIMIR duan2024membership can be found in Appendix \ref{['sec: estimation granularity']}.
• Figure 5: Results of PETAL with various surrogate models.