General Purpose Verification for Chain of Thought Prompting

TL;DR

This paper tackles the problem that chain-of-thought reasoning in LLMs can yield correct results despite flawed intermediate steps. It introduces a general-purpose, problem-agnostic verification framework with step-level verifiers for relevance, mathematical accuracy, and logical consistency, augmented by step-wise perplexity to steer generation. The authors formalize a two-component system (solution generator G and verifiers V), define notation, and present aggregation methods that combine step-level signals into chain-level scores for improved reasoning—often outperforming vanilla generation and Best-of-N baselines across nine datasets and four task types. Empirical results show consistent gains, including improved performance in Self-Consistency ensembles and partial-step verification, while human evaluation links verifier signals to human judgments, albeit with variance and energy-time trade-offs. The work demonstrates that LLMs are capable of detecting mistakes in their reasoning with a general verification framework, highlighting practical implications for robust, interpretable reasoning in open-ended tasks and framing avenues for more efficient verifiers and broader multilingual evaluation.

Abstract

Many of the recent capabilities demonstrated by Large Language Models (LLMs) arise primarily from their ability to exploit contextual information. In this paper, we explore ways to improve reasoning capabilities of LLMs through (1) exploration of different chains of thought and (2) validation of the individual steps of the reasoning process. We propose three general principles that a model should adhere to while reasoning: (i) Relevance, (ii) Mathematical Accuracy, and (iii) Logical Consistency. We apply these constraints to the reasoning steps generated by the LLM to improve the accuracy of the final generation. The constraints are applied in the form of verifiers: the model itself is asked to verify if the generated steps satisfy each constraint. To further steer the generations towards high-quality solutions, we use the perplexity of the reasoning steps as an additional verifier. We evaluate our method on 4 distinct types of reasoning tasks, spanning a total of 9 different datasets. Experiments show that our method is always better than vanilla generation, and, in 6 out of the 9 datasets, it is better than best-of N sampling which samples N reasoning chains and picks the lowest perplexity generation.

Figures (14)

• Figure 1: An illustrative example highlighting how the final answer can ultimately be correct (i.e. $12 * \$40 - (12 - 10) * \$40 * 0.05 = \$476$), but it is reached through steps that are (i) irrelevant (Step 1), (ii) contradicting previous steps (Step 4 contradicts Step 3), or (iii) with mathematical errors (Step 5).
• Figure 2: An example of each of our proposed verifiers applied to a given question and previous steps.
• Figure 3: Comparison between our proposed method and two baselines, when using Self-Consistency and between 1-10 reasoning chains. We report Accuracy ($\uparrow$).
• Figure 4: Verifying only the first X% steps of a given reasoning chain. ($\uparrow$).
• Figure 5: Correlation between the scores of our proposed verifiers and the assessment of human annotators over the three reasoning principles explored in this work. All correlations have a p-value of less than $0.0001$.