Finding the Cracks: Improving LLMs Reasoning with Paraphrastic Probing and Consistency Verification

TL;DR

The paper tackles the problem of error accumulation in LLM reasoning by identifying critical tokens that disproportionately influence subsequent steps. It proposes PPCV, a self-contained two-stage framework: Paraphrastic Probing uses paraphrased questions to locate pivotal tokens via token-level logit mismatches, and Consistency Verification substitutes these tokens to produce multiple reasoning paths and selects the final answer based on cross-paraphrase consistency, optionally weighted by paraphrase similarity. Across diverse math and knowledge benchmarks and multiple LLMs, PPCV yields substantial improvements over strong baselines, including Self-Consistency and decoding-based methods, and ablation studies confirm the necessity of both stages and the effectiveness of paraphrase-consistency over majority voting. The approach offers a practical, model-agnostic means to enhance reasoning reliability without external verifiers, with clear pathways to scalable deployment by exploiting parallelizable rollout steps. Overall, PPCV advances robust, inference-time improvements in LLM reasoning and demonstrates the value of leveraging surface-form perturbations to diagnose and rectify reasoning paths.

Abstract

Large language models have demonstrated impressive performance across a variety of reasoning tasks. However, their problem-solving ability often declines on more complex tasks due to hallucinations and the accumulation of errors within these intermediate steps. Recent work has introduced the notion of critical tokens--tokens in the reasoning process that exert significant influence on subsequent steps. Prior studies suggest that replacing critical tokens can refine reasoning trajectories. Nonetheless, reliably identifying and exploiting critical tokens remains challenging. To address this, we propose the Paraphrastic Probing and Consistency Verification~(PPCV) framework. PPCV operates in two stages. In the first stage, we roll out an initial reasoning path from the original question and then concatenate paraphrased versions of the question with this reasoning path. And we identify critical tokens based on mismatches between the predicted top-1 token and the expected token in the reasoning path. A criterion is employed to confirm the final critical token. In the second stage, we substitute critical tokens with candidate alternatives and roll out new reasoning paths for both the original and paraphrased questions. The final answer is determined by checking the consistency of outputs across these parallel reasoning processes. We evaluate PPCV on mainstream LLMs across multiple benchmarks. Extensive experiments demonstrate PPCV substantially enhances the reasoning performance of LLMs compared to baselines.

Figures (10)

• Figure 1: Comparison of the effects of critical tokens and Self-Consistency on the reasoning performance of LLMs, evaluated on samples from the GSM8K training data.
• Figure 2: Case study illustrating that LLMs can refine intermediate incorrect reasoning segments (highlighted in red).
• Figure 3: An example demonstrating how substitution of a critical token (red) with a candidate token (purple) modifies the reasoning path and produces the correct answer.
• Figure 4: Illustration of the proposed paraphrastic probing and consistency verification (PPCV) framework. The framework comprises two stages: (i) probing critical tokens through paraphrased forms, and (ii) rolling out new reasoning steps with alternative tokens and selecting the final answer using the paraphrase consistency verification mechanism.
• Figure 5: Case study illustrating the identification and effects of critical tokens identified by our method. Tokens highlighted in red indicate candidate critical tokens, whereas tokens highlighted in purple correspond to alternative tokens generated when conditioning on paraphrased questions.