Window-based Membership Inference Attacks Against Fine-tuned Large Language Models

TL;DR

This work tackles privacy risks in fine-tuned LLMs by showing that traditional membership inference attacks fail to detect sparse, localized memorization signals when relying on global loss statistics. It introduces Window-Based Comparison (WBC), a sliding-window, sign-based aggregation attack that compares per-token losses between a target fine-tuned model and a reference model across multiple window sizes and ensembles the results. Through empirical analyses and extensive experiments on eleven datasets and multiple model architectures, WBC significantly outperforms baselines in AUC and low-FPR regimes, revealing strong privacy vulnerabilities that scale with model size and persist under several defenses. The findings highlight the need for defense strategies that can disrupt localized memorization patterns, and the work provides a practical, reproducible framework for evaluating such defenses in real-world settings.

Abstract

Most membership inference attacks (MIAs) against Large Language Models (LLMs) rely on global signals, like average loss, to identify training data. This approach, however, dilutes the subtle, localized signals of memorization, reducing attack effectiveness. We challenge this global-averaging paradigm, positing that membership signals are more pronounced within localized contexts. We introduce WBC (Window-Based Comparison), which exploits this insight through a sliding window approach with sign-based aggregation. Our method slides windows of varying sizes across text sequences, with each window casting a binary vote on membership based on loss comparisons between target and reference models. By ensembling votes across geometrically spaced window sizes, we capture memorization patterns from token-level artifacts to phrase-level structures. Extensive experiments across eleven datasets demonstrate that WBC substantially outperforms established baselines, achieving higher AUC scores and 2-3 times improvements in detection rates at low false positive thresholds. Our findings reveal that aggregating localized evidence is fundamentally more effective than global averaging, exposing critical privacy vulnerabilities in fine-tuned LLMs.

Figures (7)

• Figure 1: Performance comparison of membership inference attacks on Pythia-2.8B. ROC curves showing true positive rate vs. false positive rate (both on log scale) for various MIA methods evaluated on Web Samples V2 split. The diagonal dashed line represents random guessing performance. Numbers in legends indicate AUC scores. Our proposed WBC Attack significantly outperforms existing baselines across all false positive rate regimes, demonstrating superior membership inference capability.
• Figure 2: Overview of the Window-Based Comparison (WBC) Attack. Unlike baseline methods (a) that rely on comparing a single, noisy global average of per-token losses, our approach (b) introduces a local aggregation step. We slide a window across the loss sequences from the target ($\mathcal{M}^\text{T}$) and reference ($\mathcal{M}^\text{R}$) models, making a binary comparison for each window. The final membership score is the sum of this local evidence, a process that filters noise and provides a more sensitive measure, leading to better separation between member and non-member distributions.
• Figure 3: Empirical distribution of token-level loss differences. (a) Log-scale density plots reveal long-tailed distributions with a subtle rightward shift for members. The difference is most pronounced in the left tail, not the mean. (b) Complementary CDF confirms the long-tailed behavior of the token loss. (c) Time series shows sparse, scattered extremes.
• Figure 4: Window size trade-off. Performance of single-window WBC attacks as a function of window size $w \in [1, 39]$ on six datasets. Metrics include AUC, TPR@10% FPR, and TPR@1% FPR. Shaded regions show the standard deviation over 100 bootstrap samples.
• Figure 5: Aggregation method comparison across datasets. (a) AUC performance across four aggregation strategies. (b,c) TPR at low FPR thresholds shows amplified advantages in high-precision regimes.