Benchmark Data Contamination of Large Language Models: A Survey

TL;DR

Benchmark Data Contamination (BDC) undermines the reliability of LLM evaluations by leaking benchmark information into training data. The paper provides a comprehensive survey that splits detection methods into matching-based and comparison-based techniques, and mitigation strategies into data curation, data refactoring, and benchmark-free evaluation, highlighting tools such as TS-Guessing, EvoEval, TreeEval, and FreeEval. It discusses practical challenges like data access, computational cost, and semantic contamination, and suggests future directions including dynamic benchmarks, human-in-the-loop evaluation, and content-tagging for benchmarks. The work argues for a multi-faceted evaluation framework to preserve trustworthy assessments as LLMs and their training data continue to grow in scale and complexity.

Abstract

The rapid development of Large Language Models (LLMs) like GPT-4, Claude-3, and Gemini has transformed the field of natural language processing. However, it has also resulted in a significant issue known as Benchmark Data Contamination (BDC). This occurs when language models inadvertently incorporate evaluation benchmark information from their training data, leading to inaccurate or unreliable performance during the evaluation phase of the process. This paper reviews the complex challenge of BDC in LLM evaluation and explores alternative assessment methods to mitigate the risks associated with traditional benchmarks. The paper also examines challenges and future directions in mitigating BDC risks, highlighting the complexity of the issue and the need for innovative solutions to ensure the reliability of LLM evaluation in real-world applications.

Figures (7)

• Figure 1: An illustration of the method developed by deng2023investigating for identifying BDC in modern benchmarks. The figure on the left shows the workflow of an information retrieval system, which aims to detect potentially contaminated data within a benchmark by utilizing a pre-trained corpus. The figure on the right introduces TS-Guessing, an approach for detecting potential contamination. This technique involves concealing parts of the information in the test set and prompting LLMs to infer the missing elements. If the LLMs can accurately predict the same missing option as the one in the test set, it raises the suspicion that they may have encountered the benchmark data during their training.
• Figure 2: The scheme proposed by chandran2024private.
• Figure 3: Overview of EVOEVAL evolving problem generation pipeline proposed by xia2024leaderboard
• Figure 4: The Meta Probing Agent (MPA) zhu2024dyval process that transforms an original benchmark into a new one. The principles here can be combined to create various probing benchmarks for multifaceted analysis. In (c) we see how MPA generates a new sample, given an existing sample from ARC-C clark2018think.
• Figure 5: Auto-dataset update framework proposed by ying2024seen, who deployed two strategies: mimicking and extending to update.

Theorems & Definitions (1)

• definition 1: Benchmark Data Contamination