Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Models

Shahriar Golchin, Mihai Surdeanu

TL;DR

The paper tackles data contamination in LLMs by introducing DCQ, a quiz-based, fully black-box approach that detects and estimates verbatim memorization without access to training data or model weights. It frames detection as selecting the original dataset instance among word-level perturbed variants, using a Bias Detector Quiz to reveal positional biases and a Bias Compensator Quiz to robustly estimate contamination through permutation. The method yields contamination estimates that surpass replication-based approaches and can bypass safety filters, providing a scalable, task- and language-agnostic tool for real-world evaluations. This work advances benchmark integrity by turning memorization signals into quantitative contamination measurements, with practical implications for model evaluation and dataset governance.

Abstract

We propose the Data Contamination Quiz (DCQ), a simple and effective approach to detect data contamination in large language models (LLMs) and estimate the amount of it. Specifically, we frame data contamination detection as a series of multiple-choice questions, devising a quiz format wherein three perturbed versions of each instance, subsampled from a specific dataset partition, are created. These changes only include word-level perturbations. The generated perturbations, along with the original dataset instance, form the options in the DCQ, with an extra option accommodating the selection of none of the provided options. Given that the only distinguishing signal among the options is the exact wording with respect to the original dataset instance, an LLM, when tasked with identifying the original dataset instance, gravitates towards selecting the original one if it has been exposed to it. While accounting for positional biases in LLMs, the quiz performance reveals the contamination level for the tested model with the dataset partition to which the quiz pertains. Applied to various datasets and LLMs, under controlled and uncontrolled contamination, our findings, while fully lacking access to training data and model parameters, suggest that DCQ achieves state-of-the-art results and uncovers greater contamination levels through memorization compared to existing methods. Also, it proficiently bypasses more safety filters, especially those set to avoid generating copyrighted content.

Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Models

TL;DR

The paper tackles data contamination in LLMs by introducing DCQ, a quiz-based, fully black-box approach that detects and estimates verbatim memorization without access to training data or model weights. It frames detection as selecting the original dataset instance among word-level perturbed variants, using a Bias Detector Quiz to reveal positional biases and a Bias Compensator Quiz to robustly estimate contamination through permutation. The method yields contamination estimates that surpass replication-based approaches and can bypass safety filters, providing a scalable, task- and language-agnostic tool for real-world evaluations. This work advances benchmark integrity by turning memorization signals into quantitative contamination measurements, with practical implications for model evaluation and dataset governance.

Abstract

We propose the Data Contamination Quiz (DCQ), a simple and effective approach to detect data contamination in large language models (LLMs) and estimate the amount of it. Specifically, we frame data contamination detection as a series of multiple-choice questions, devising a quiz format wherein three perturbed versions of each instance, subsampled from a specific dataset partition, are created. These changes only include word-level perturbations. The generated perturbations, along with the original dataset instance, form the options in the DCQ, with an extra option accommodating the selection of none of the provided options. Given that the only distinguishing signal among the options is the exact wording with respect to the original dataset instance, an LLM, when tasked with identifying the original dataset instance, gravitates towards selecting the original one if it has been exposed to it. While accounting for positional biases in LLMs, the quiz performance reveals the contamination level for the tested model with the dataset partition to which the quiz pertains. Applied to various datasets and LLMs, under controlled and uncontrolled contamination, our findings, while fully lacking access to training data and model parameters, suggest that DCQ achieves state-of-the-art results and uncovers greater contamination levels through memorization compared to existing methods. Also, it proficiently bypasses more safety filters, especially those set to avoid generating copyrighted content.
Paper Structure (16 sections, 5 figures, 6 tables)

This paper contains 16 sections, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Approach
  3. Creating Quiz Options
  4. Detecting and Handling Positional Biases
  5. Estimating Contamination Levels by Quiz Performance
  6. Experimental Setup
  7. Empirical Results and Evaluation
  8. Positional Adversarial Analysis
  9. Evaluation under Controlled Contamination
  10. Evaluation in the Wild
  11. Related Work
  12. Conclusion
  13. Detecting Data Contamination for Different Checkpoints
  14. Details and Analysis of Generated Word-Level Perturbations
  15. Systematic Contamination of GPT-3.5 and Llama 2
  16. ...and 1 more sections

Figures (5)

  • Figure 1: An example of a quiz question crafted to detect data contamination within the test partition of the XSum dataset. Here, the produced answer by the underlying LLM (GPT-4) aligns with the correct option (option C), signaling previous exposure to data, and thus, revealing contamination.
  • Figure 2: Results of positional adversarial analysis. The green bar charts show the selection frequency distribution of options in BDQ while the blue and red ones represent results after replacing new word-level perturbations and original dataset instances with the least-chosen option detected by BDQ, respectively. The increase in selection frequency for options containing original dataset instances (red bar charts) signals the model's prior exposure to data. In all settings, GPT-4 is the base model.
  • Figure 3: An illustration of the full process for our approach, performed on the AG News train set. Initially, Bias Detector Quiz (BDQ) is executed to identify non-preferred options (i.e., B, C, and D), followed by multiple Bias Compensator Quizzes (BCQs) to determine the maximum contamination (88%) through permutation among the non-preferred options. In all settings, GPT-4 is the base model.
  • Figure 4: The zero-shot prompt employed for generating four word-level perturbations per dataset instance. The input text is replaced for each dataset instance, and GPT-4 is prompted to generate the four perturbations for each dataset instance individually at once. The format provided in the input prompt is adjusted based on a specific dataset/task, with column names in the dataset being used to denote distinct components of a dataset instance. For example, "Summary" is a column name in the XSum that is prepended to the instance shown above. The example shown here is an instance taken from the XSum test partition along with its generated word-level perturbations, as illustrated earlier in Figure \ref{['figure:example-of-contamination-quiz']}.
  • Figure 5: Percentage distribution of evaluation results for meaning and sentence structure of the generated word-level perturbations based on majority voting among three expert raters. The most common evaluation is good for both studied criteria.