Automatic Calibration for Membership Inference Attack on Large Language Models

TL;DR

Automatic Calibration Membership Inference Attack (ACMIA) introduces a tunable temperature to reshape token-output probabilities and calibrate scores for MIA on large language models, addressing high false positives and external-reference dependencies. By deriving a Temperature-Scaled Probability (TSP) and offering three variants (AC, DerivAC, NormAC), ACMIA achieves state-of-the-art AUROC and TPR@5%FPR across WikiMIA, MIMIR, and PatentMIA on multiple open-source LLMs without requiring external reference models. The method is supported by theoretical grounding in implicit score matching and practical ablations showing robustness to temperature, model size, and text length; AC/DerivAC work with limited access, while NormAC leverages full log-prob distributions. Overall, ACMIA provides a principled, practical red-teaming tool for privacy auditing in LLMs with broad applicability and minimal external dependencies.

Abstract

Membership Inference Attacks (MIAs) have recently been employed to determine whether a specific text was part of the pre-training data of Large Language Models (LLMs). However, existing methods often misinfer non-members as members, leading to a high false positive rate, or depend on additional reference models for probability calibration, which limits their practicality. To overcome these challenges, we introduce a novel framework called Automatic Calibration Membership Inference Attack (ACMIA), which utilizes a tunable temperature to calibrate output probabilities effectively. This approach is inspired by our theoretical insights into maximum likelihood estimation during the pre-training of LLMs. We introduce ACMIA in three configurations designed to accommodate different levels of model access and increase the probability gap between members and non-members, improving the reliability and robustness of membership inference. Extensive experiments on various open-source LLMs demonstrate that our proposed attack is highly effective, robust, and generalizable, surpassing state-of-the-art baselines across three widely used benchmarks. Our code is available at: \href{https://github.com/Salehzz/ACMIA}{\textcolor{blue}{Github}}.

Figures (5)

• Figure 1: Illustration of our ACMIA vs. existing methods: Let $x_1$ and $x_2$ represent member (training) texts, and $x_3$ and $x_4$ represent non-member (non-training) texts. Existing methods typically rely on the output log probabilities of input tokens to determine membership, which can result in both false positives (e.g., non-member $x_3$ is classified as member), and false negatives (e.g., member $x_2$ is classified as non-member). In contrast, our proposed ACMIA method first reshapes the log-likelihood distribution using a tunable temperature $\tau$ to enhance the separation between members and non-members. It then calibrates the scores to allow a fair comparison between simple and complex samples, regardless of their membership status. Specifically, among the member texts, $x_1$ represents a simpler example while $x_2$ is more complex; similarly, $x_3$ and $x_4$ are non-member texts, with $x_3$ being simpler and $x_4$ more complex. By adjusting for these differences, our method reduces both false positives and false negatives.
• Figure 2: The three histograms show the density distributions of normalized scores for members (blue) and non-members (orange) across different methods: (left) Log-likelihood scores, (middle) Min-$K$% scores, and (right) our automatic calibrated scores. The numerical difference between member and non-member means is indicated in each plot, showing that our approach achieves the largest separation, improving the reliability of membership inference.
• Figure 3: AUROC with different temperatures on WikiMIA using NeoX-20B and PatentMIA using Qwen1.5-32B. Our ACMIA framework, including AC, DerivAC, and NormAC, is fairly robust to the tunable temperatures.
• Figure 4: AUROC with different sizes of Qwen1.5 on PatentMIA.
• Figure 5: AUROC with different text lengths on PatentMIA dataset.