Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework

TL;DR

This work challenges the assumption that PII leakage in LLMs directly reflects memorization by introducing Cue-Resistant Memorization (CRM), a cue-controlled framework that conditions memorization metrics on prompt–target overlap. Through a large multilingual study (32 languages) across verbatim, associative, cue-free, and membership-inference paradigms, the authors show that most reported leakage arises from surface cues in prompts rather than genuine memorization, and that true memorization signals vanish under low-cue conditions. They formalize metrics such as $HR(\tau)$, $Recon(\tau)$, and overlap cue $c(s,p)$, and demonstrate that reconstruction probabilities and hits are highly cue-driven, with cue-free generation yielding negligible PII leakage and MIAs performing near random. The study provides a principled, generalizable method for evaluating privacy-relevant memorization in LLMs and highlights the need for cue-controlled evaluation to avoid overestimating privacy risks in real-world deployments.

Abstract

Large Language Models (LLMs) have been reported to "leak" Personally Identifiable Information (PII), with successful PII reconstruction often interpreted as evidence of memorization. We propose a principled revision of memorization evaluation for LLMs, arguing that PII leakage should be evaluated under low lexical cue conditions, where target PII cannot be reconstructed through prompt-induced generalization or pattern completion. We formalize Cue-Resistant Memorization (CRM) as a cue-controlled evaluation framework and a necessary condition for valid memorization evaluation, explicitly conditioning on prompt-target overlap cues. Using CRM, we conduct a large-scale multilingual re-evaluation of PII leakage across 32 languages and multiple memorization paradigms. Revisiting reconstruction-based settings, including verbatim prefix-suffix completion and associative reconstruction, we find that their apparent effectiveness is driven primarily by direct surface-form cues rather than by true memorization. When such cues are controlled for, reconstruction success diminishes substantially. We further examine cue-free generation and membership inference, both of which exhibit extremely low true positive rates. Overall, our results suggest that previously reported PII leakage is better explained by cue-driven behavior than by genuine memorization, highlighting the importance of cue-controlled evaluation for reliably quantifying privacy-relevant memorization in LLMs.

Figures (13)

• Figure 1: Cross-lingual "PII associative reconstruction" in mGPT3-13B is driven by strong cues (e.g., names and common email patterns), enabling email inference across languages regardless of train/test membership and indicating cue-driven generalization rather than memorization.
• Figure 2: Email and Phone verbatim CRM hit rates $\operatorname{HR}(\tau)$ under different cue thresholds $\tau$. For reference, the average cues of Email is 0.50, Phone is 0.18.
• Figure 3: Per-language CRM hit rates $\operatorname{HR}(\tau)$ for verbatim Phone and Email memorization under different cue thresholds $\tau$, using the email twins template for mGPT3-13B.
• Figure 4: Per-language CRM hit rates $\operatorname{HR}(\tau)$ for associative PII memorization under different cue thresholds $\tau$ using the email twins template for mGPT3-13B. The average cue overlap is 0.36.
• Figure 5: Log-likelihood under cues interval for verbatim (left) and associative (right) reconstruction settings for mGPT3-13B. Reconstruction probability is averaged over disjoint cue intervals of width 0.1, exhibits a strong positive correlation with overlap cues and follows highly similar trends for training and test samples.