ConStat: Performance-Based Contamination Detection in Large Language Models

TL;DR

ConStat tackles the reliability problem of benchmark contamination in large language models by redefining contamination as artificially inflated, non-generalizing performance. It introduces a statistically grounded test that compares a target model’s performance on an original benchmark to its performance on a reference benchmark, using a hardness-correction function derived from a diverse set of reference models. The method bootstraps over reference models and samples to estimate contamination magnitude ${\delta}$ and a p-value, without requiring access to training data. Extensive controlled experiments and broad-scale analyses reveal substantial contamination in several popular models and top leaderboards, highlighting the need for principled, performance-based evaluation frameworks in AI. ConStat thus offers a robust, generalizable tool for quantifying and mitigating benchmark contamination, with direct implications for fair model comparison and benchmark design.

Abstract

Public benchmarks play an essential role in the evaluation of large language models. However, data contamination can lead to inflated performance, rendering them unreliable for model comparison. It is therefore crucial to detect contamination and estimate its impact on measured performance. Unfortunately, existing detection methods can be easily evaded and fail to quantify contamination. To overcome these limitations, we propose a novel definition of contamination as artificially inflated and non-generalizing benchmark performance instead of the inclusion of benchmark samples in the training data. This perspective enables us to detect any model with inflated performance, i.e., performance that does not generalize to rephrased samples, synthetic samples from the same distribution, or different benchmarks for the same task. Based on this insight, we develop ConStat, a statistical method that reliably detects and quantifies contamination by comparing performance between a primary and reference benchmark relative to a set of reference models. We demonstrate the effectiveness of ConStat in an extensive evaluation of diverse model architectures, benchmarks, and contamination scenarios and find high levels of contamination in multiple popular models including Mistral, Llama, Yi, and the top-3 Open LLM Leaderboard models.

Figures (3)

• Figure 1: Overview of our method. We first select models to check for contamination, then select reference models and benchmarks, and finally compute ConStat to detect and quantify contamination.
• Figure 2: Estimated $\hat{\delta}$ as a function of the true $\delta$ for the finetuned models. $2$-sigma intervals are shown.
• Figure 3: CDF of various statistical tests for uncontaminated models in different scenarios.

Theorems & Definitions (1)

• Definition 1: $\delta$-Contamination