Pessimistic Verification for Open Ended Math Questions

TL;DR

Pessimistic verification addresses the challenge of reliably verifying open-ended mathematical reasoning by introducing multi-pass verification variants. The three methods Simple, Vertical, and Progressive run parallel checks and deem a proof incorrect if any pass detects an error, improving error-detection rates without heavy resource use. Experiments on Hard2Verify, IMO-GradingBench, and QiuZhen-Bench show that Progressive verification delivers the best balance of accuracy and efficiency, approaching human expert reliability while revealing annotation issues in stronger models. The work emphasizes that error-centered verification can strengthen long-horizon math tasks and suggests integrating such verifiers into training and benchmarking pipelines.

Abstract

The key limitation of the verification performance lies in the ability of error detection. With this intuition we designed several variants of pessimistic verification, which are simple workflows that could significantly improve the verification of open-ended math questions. In pessimistic verification we construct multiple parallel verifications for the same proof, and the proof is deemed incorrect if any one of them reports an error. This simple technique significantly improves the performance across many math verification benchmarks without incurring substantial computational resources. Its token efficiency even surpassed extended long-CoT in test-time scaling. Our case studies further indicate that the majority of false negatives in stronger models are actually caused by annotation errors in the original dataset, so our method's performance is in fact underestimated. Self-verification for mathematical problems can effectively improve the reliability and performance of language model outputs, and it also plays a critical role in enabling long-horizon mathematical tasks. We believe that research on pessimistic verification will help enhance the mathematical capabilities of language models across a wide range of tasks.

Figures (6)

• Figure 1: The response-level performance and efficiency of pessimistic verification methods on GPT-5-mini. The equivalent output tokens is calculated by weighting the model's input and output token prices. For GPT-5-mini it is (input_tokens / 8 + output_tokens). These results indicate that pessimistic verification has stable and efficient scalability.
• Figure 2: Three variants of pessimistic verification methods in this work. In following experiments they will separately be denoted as "pes@n", "vp@l", and "prog@n/l"
• Figure 3: The comparion of simple pessimistic verification and majority voting. The former exhibits steady performance gains as sampling budget increases, whereas the latter shows almost no changes in performance.
• Figure 4: The vertical review prompting method used in vertical pessimistic verification and progressive pessimistic verification.
• Figure 5: The response-level performance and efficiency of three pessimistic verification methods on Qwen3-30B-A3B.