Membership Inference on LLMs in the Wild

TL;DR

This work addresses auditing training data in large language models by formulating a robust black-box membership inference attack called SimMIA, coupled with a new benchmark WikiMIA-25 for modern proprietary LLMs. SimMIA uses a word-by-word sampling strategy to avoid drift, a soft semantic scoring mechanism based on word embeddings, and a relative aggregation scheme to produce a membership score, achieving state-of-the-art results in black-box settings and rivaling gray-box approaches. The authors demonstrate strong cross-benchmark generalization, including substantial gains on WikiMIA-25 and resilience across model families, while also examining ablations and the impact of prefix length, sample size, and hyperparameters. They also discuss progressive disclosure scenarios and provide guidance for auditing contemporary LLMs, while acknowledging sampling overhead as a limitation and outlining directions for improving sample efficiency.

Abstract

Membership Inference Attacks (MIAs) act as a crucial auditing tool for the opaque training data of Large Language Models (LLMs). However, existing techniques predominantly rely on inaccessible model internals (e.g., logits) or suffer from poor generalization across domains in strict black-box settings where only generated text is available. In this work, we propose SimMIA, a robust MIA framework tailored for this text-only regime by leveraging an advanced sampling strategy and scoring mechanism. Furthermore, we present WikiMIA-25, a new benchmark curated to evaluate MIA performance on modern proprietary LLMs. Experiments demonstrate that SimMIA achieves state-of-the-art results in the black-box setting, rivaling baselines that exploit internal model information.

Figures (3)

• Figure 1: Comparison between SaMIA samia, a representative black-box MIA baseline, and our proposed SimMIA. Given a text $x$: (a) SaMIA samples multiple candidate continuations conditioned on the underlined prefix, computes ROUGE-N scores lin-2004-rouge between each candidate and $x$, and aggregates these scores to determine membership; (b) SimMIA instead performs word-by-word sampling for every prefix, computes word-level scores based on sampled and the corresponding words from $x$ using word embeddings, and aggregates these scores to derive the final membership prediction. denotes the empirical frequency of the corresponding word, while is the empirical frequency if additionally conditioning on non-member data.
• Figure 2: Relative score distributions for members and non-members. The relative score is the ratio of the perturbed score to the unperturbed score in Eq. \ref{['eqn:word-score']}. Consistent with recall, members show lower relative scores compared to non-members.
• Figure 3: WikiMIA (length 32) results under various prefix ratios, sample sizes, and non-member shots.