Benchmarks Saturate When The Model Gets Smarter Than The Judge

TL;DR

Benchmarks saturate when models approach judge-level competence, and Omni-MATH-2 reveals that both dataset quality and judge reliability shape evaluated performance, not just model ability. The authors create Omni-MATH-2 with an exact-answer-judgable Filtered subset ($n=4{,}181$) and a tagged non-standard subset ($n=247$), enabling explicit measurement of dataset-induced and judge-induced errors. Across five state-of-the-art models, judge choice dramatically shifts results, and expert audits show Omni-Judge is wrong in the majority of disagreements, even on clean items, highlighting evaluation bottlenecks as models near saturation. The work advocates treating benchmarks as dataset-model-judge triplets, emphasizes dataset audits and judge calibration, and promotes multi-judge evaluation and uncertainty reporting to avoid misinterpreting model capabilities.

Abstract

Benchmarks are important tools to track progress in the development of Large Language Models (LLMs), yet inaccuracies in datasets and evaluation methods consistently undermine their effectiveness. Here, we present Omni-MATH-2, a manually revised version of the Omni-MATH dataset comprising a clean, exact-answer subset ($n{=}4181$) and a tagged, non-standard subset ($n{=}247$). Each problem was audited to ensure LaTeX compilability, solvability and verifiability, which involved adding missing figures or information, labeling problems requiring a proof, estimation or image, and removing clutter. This process significantly reduces dataset-induced noise, thereby providing a more precise assessment of model performance. The annotated dataset also allows us to evaluate judge-induced noise by comparing GPT-5 mini with the original Omni-Judge, revealing substantial discrepancies between judges on both the clean and tagged problem subsets. Expert annotations reveal that Omni-Judge is wrong in $96.4\%$ of the judge disagreements, indicating its inability to differentiate between models' abilities, even well before saturation of the benchmark occurs. As problems become more challenging, we find that increasingly competent judges become essential in order to prevent judge errors from masking genuine differences between models. Finally, neither judge identifies the present failure modes for the subset of tagged problems, demonstrating that dataset quality and judge reliability are both critical to develop accurate benchmarks of model performance.

Figures (10)

• Figure 1: Overview of the cleaning process of the Omni-MATH dataset gao2024omni. First, we check the compilability of the $4{,}428$ problem statements in LaTeX and convert them to valid LaTeX code with python. Next, a PhD-level mathematician manually went through the compiled pdf files twice, checking for the solvability and verifiability of each problem. In this process, missing information was added when available through browsing manually or through GPT-5.1, and images where added to the data folder. Furthermore, a tag was added to each problem containing an image, proof or estimation. Degenerate problems received the should delete tag. We refer to the resulting dataset as Omni-MATH-2, which contains the same number of entries as the original Omni-MATH dataset (647 edited problems ($14.6\%$), 247 tagged as non-standard ($5.6\%$), see Table \ref{['tab:Table1']}). The Omni-MATH-2-Filtered dataset ($n{=}4{,}181$) is the subset of cleaned questions, excluding the tagged ones. This makes it suitable for exact-answer judges.
• Figure 2: Example of the evaluation process for a question labelled as image. The original problem in Omni-MATH misses the corresponding image, rendering the problem unsolvable. The model to be evaluated, GPT-5, correctly identifies that there is missing information, but Omni-Judge counts this as an incorrect answer.
• Figure 3: The judge-induced difference in accuracy on Omni-MATH-2-Filtered is not uniformly distributed across disciplines, difficulty tiers or models, indicating structural evaluation noise rather than i.i.d. label noise. Evaluating five state-of-the-art models on Omni-MATH-2-Filtered—Claude Sonnet 4.5, DeepSeek v3.2, Gemini 3 Pro, GPT-5 and Kimi K2 Thinking—produced a different ranking of their mathematical abilities when evaluated with GPT-5 Mini rather than with Omni-Judge, which calls into question the interpretability of the inter-model differences. Furthermore, the right-hand panels show that, as questions become more difficult, judge disagreement increases, indicating that the judge's conclusion is more important for challenging problems. The difference between judges' answers is also model- and domain-dependent, with some disciplines and models producing larger differences than others (e.g. Claude and Deepseek, Calculus domain). For numerical performance scores with Bayesian confidence intervals consult \ref{['tab:Table3']}.
• Figure 4: Example of a judge disagreement between Omni-Judge and GPT-5 mini. Here, it is not straightforward to see whether the model's answer is equivalent to the reference answer. Two expert mathematicians, with the help of an LLM council, building on Claude Opus 4.5, DeepSeek v3.2 speciale, DeepSeek v3.2, GPT-5, and Gemini 3 pro, verified that the model's answer is in fact correct and equivalent to the reference answer. GPT-5 mini thus correctly assesses equivalence.
• Figure 5: On the subset of estimation problems in Omni-MATH-2, Omni-Judge and GPT-5 mini evaluate a substantial portion of strong estimates as incorrect. We compute the contest score by using the scoring rule present in the problem statement or solution.