Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Detecting Non-Membership in LLM Training Data via Rank Correlations

Pranav Shetty, Mirazul Haque, Zhiqiang Ma, Xiaomo Liu

Abstract

As large language models (LLMs) are trained on increasingly vast and opaque text corpora, determining which data contributed to training has become essential for copyright enforcement, compliance auditing, and user trust. While prior work focuses on detecting whether a dataset was used in training (membership inference), the complementary problem -- verifying that a dataset was not used -- has received little attention. We address this gap by introducing PRISM, a test that detects dataset-level non-membership using only grey-box access to model logits. Our key insight is that two models that have not seen a dataset exhibit higher rank correlation in their normalized token log probabilities than when one model has been trained on that data. Using this observation, we construct a correlation-based test that detects non-membership. Empirically, PRISM reliably rules out membership in training data across all datasets tested while avoiding false positives, thus offering a framework for verifying that specific datasets were excluded from LLM training.

Detecting Non-Membership in LLM Training Data via Rank Correlations

Abstract

As large language models (LLMs) are trained on increasingly vast and opaque text corpora, determining which data contributed to training has become essential for copyright enforcement, compliance auditing, and user trust. While prior work focuses on detecting whether a dataset was used in training (membership inference), the complementary problem -- verifying that a dataset was not used -- has received little attention. We address this gap by introducing PRISM, a test that detects dataset-level non-membership using only grey-box access to model logits. Our key insight is that two models that have not seen a dataset exhibit higher rank correlation in their normalized token log probabilities than when one model has been trained on that data. Using this observation, we construct a correlation-based test that detects non-membership. Empirically, PRISM reliably rules out membership in training data across all datasets tested while avoiding false positives, thus offering a framework for verifying that specific datasets were excluded from LLM training.
Paper Structure (35 sections, 13 equations, 7 figures, 12 tables)

This paper contains 35 sections, 13 equations, 7 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Problem setup
  5. Reference LLM.
  6. Min-K%++
  7. Empirical motivation for PRISM
  8. Methods
  9. Comparing Rank Correlations
  10. Training $M_{D}$
  11. Hypothesis Testing via Rank-Correlation Comparisons
  12. Test statistic.
  13. Bootstrap inference.
  14. Test for evidence of non-training on $\mathcal{D}$ (non-membership).
  15. Results
  16. ...and 20 more sections

Figures (7)

  • Figure 1: Overview of PRISM. Logits of Reference LLM, Target LLM, and Distilled Reference LLMs are used to calculate Min-K%++ scores, and the corresponding ranking of the scores. Final statistical testing for non-membership detection is dependent on the rank correlation between the reference and target model, and the rank correlation between the distilled reference and target model.
  • Figure 2: Overview of distilling the reference model. The distillation is dependent on two components: cross entropy of the reference model and KL divergence between the output probability of the target and reference model.
  • Figure 3: p-value for non-member detection when different number of samples are used
  • Figure 4: a) Spearman rank correlation difference of Pythia-410m and Pythia-410m-CPT against Pythia-1b reference model at different values of $K$ in Min-K%++ b) Change in rank of a document when Min-K%++ is computed using Pythia-410m-CPT relative to the rank of the Pythia-410m model.
  • Figure 5: Hyperparameter study to determine the value of a) $\tau$ and b) $\lambda$. Pythia-410m and Pythia-410m-CPT are the target models while Pythia-1b is the reference model.
  • ...and 2 more figures