Can Language Models Falsify? Evaluating Algorithmic Reasoning with Counterexample Creation

TL;DR

This work reframes evaluation of language models by introducing REFUTE, a dynamically updated benchmark that tests the ability to falsify subtly incorrect algorithmic solutions through counterexample creation validated by code execution. It formalizes falsification as finding inputs that satisfy problem constraints but cause an incorrect solution to fail, and it analyzes state-of-the-art models, showing a substantial gap between problem-solving and counterexample generation. Empirical results reveal that even with code-execution feedback, best models solve only about half the problems yet find counterexamples for fewer than 9%, underscoring the need for stronger reflective reasoning and verification capabilities. The paper argues that improving counterexample creation can enhance model reliability, inform self-improvement, and accelerate scientific progress, while proposing future directions toward broader inverse benchmarks and integration with formal verification tools.

Abstract

There is growing excitement about the potential of Language Models (LMs) to accelerate scientific discovery. Falsifying hypotheses is key to scientific progress, as it allows claims to be iteratively refined over time. This process requires significant researcher effort, reasoning, and ingenuity. Yet current benchmarks for LMs predominantly assess their ability to generate solutions rather than challenge them. We advocate for developing benchmarks that evaluate this inverse capability - creating counterexamples for subtly incorrect solutions. To demonstrate this approach, we start with the domain of algorithmic problem solving, where counterexamples can be evaluated automatically using code execution. Specifically, we introduce REFUTE, a dynamically updating benchmark that includes recent problems and incorrect submissions from programming competitions, where human experts successfully identified counterexamples. Our analysis finds that the best reasoning agents, even OpenAI o3-mini (high) with code execution feedback, can create counterexamples for only <9% of incorrect solutions in REFUTE, even though ratings indicate its ability to solve up to 48% of these problems from scratch. We hope our work spurs progress in evaluating and enhancing LMs' ability to falsify incorrect solutions - a capability that is crucial for both accelerating research and making models self-improve through reliable reflective reasoning.

Figures (7)

• Figure 1: While standard benchmarks for algorithmic reasoning require models to generate solutions, we propose an inverse benchmark to evaluate reasoning about correctness by falsifying incorrect solutions. To allow expressivity, we let the model output a code that generates the counterexample input, and validate it by comparing the output of the incorrect solution with a held out correct solution.
• Figure 2: (Left) Sample Filtering Pipeline. We curated (problem, incorrect solution) pairs from Codeforces (Jan 2024–Jan 2025) where counterexamples are easy to verify but non-trivial to generate, using a 4-step filtering process (dynamically expandable over time). (Right) Dataset Topic Coverage. The dataset covers 35+ algorithmic topics, including many niche ones. Problems are multi-tagged—e.g., the high number of "greedy" problems indicates it’s one of the key concepts, not the only one.
• Figure 3: (Left) Distribution of Test Cases Passed by Incorrect Solutions: The median test cases passed is 17, making the mistakes non-trivial. (Center) Distribution of Problem Ratings: The benchmark spans a range of difficulty levels. (Right) Distribution of Incorrect Solution Author Ratings: Preference was given to expert authors, rated above 2000, resulting in a noticeable peak.
• Figure 4: Analysis of R1, o3-mini (high) success rate at creating counterexamples, combining success across strategies like prompting, agent and programmatic search. We find that model successes are not predictable based on problem difficulty, or attributes of the incorrect submission like author expertise and number of test cases passed before giving a wrong answer. Success rate on the hacked subset, where counterexamples were found despite passing all initial tests is 4% $\pm 1\%$.
• Figure 5: LiveCodeBench leaderboard in Feburary 2025, which we used to select the final 5 models used for our benchmarking. We took the top 3 reasoning models from unique developers, highlighted in red, and the top 2 chat models highlighted in yellow.