Lost at the Beginning of Reasoning

TL;DR

This work investigates long-chain-of-thought reasoning in large language models and finds that the initial reasoning step $t_1$ dominantly shapes the final prediction, with incorrect first steps causing large accuracy drops and extensive subsequent reasoning (overthinking). It introduces an efficient sampling method that generates multiple candidate first steps, scores them with a reward model, and continues only the top $M$ steps, achieving up to $70\%$ reduction in inference cost without sacrificing accuracy. The approach is validated across five model families and five challenging benchmarks in math, science, and programming, demonstrating robust first-step influence and practical gains. The findings suggest that optimizing or leveraging the very first step is a fruitful direction for building more accurate and compute-efficient reasoning LLMs.

Abstract

Recent advancements in large language models (LLMs) have significantly advanced complex reasoning capabilities, particularly through extended chain-of-thought (CoT) reasoning that incorporates mechanisms such as backtracking, self-reflection, and self-correction. Despite these developments, the self-correction abilities of LLMs during long CoT reasoning remain underexplored. And recent findings on overthinking suggest that such models often engage in unnecessarily redundant reasoning. In this work, we empirically show that the first reasoning step exerts a disproportionately large influence on the final prediction. I.e., errors introduced at this stage can substantially degrade subsequent reasoning quality. This phenomenon is consistently observed across various state-of-the-art open- and closed-source reasoning models. Leveraging this insight, we propose an efficient sampling strategy that leverages a reward model to identify and retain high-quality first reasoning steps while discarding suboptimal ones, achieving up to a 70% reduction in inference cost without sacrificing any accuracy. Our work highlights the central role of the first reasoning step in generating a high-quality reasoning trajectory, and thus enabling significantly efficient sampling.

Figures (12)

• Figure 1: Overview of our observation and efficient sampling. The first reasoning step $t_1$ heavily shapes the entire reasoning trajectory: a strong first step typically yields accurate solutions with fewer tokens (bottom left). Building on this observation, we propose to generate multiple candidate first steps, evaluate them with a reward model, and discard weaker candidates early (top right). This method maintains accuracy while substantially reducing token consumption by 70% (bottom right).
• Figure 2: Cosine similarity between the embeddings of the $i$-th reasoning step $t_i$ and the final conclusion $c_T$, using the average number of reasoning steps for interpolation. The reasoning steps are segmented either by GPT-5 (left) or by heuristic rules (right). See Figure \ref{['fig:semantic_similarity_longest']} for results based on the maximum number of reasoning steps used for interpolation.
• Figure 3: Accuracy and number of tokens on DS-R1-Qwen-7B. Left: The relationship between the accuracy of the final prediction ($q_T$) and the correctness of the prediction solely based on the first reasoning step ($q_1$) across different difficulty levels. If $q_1$ is incorrect, $q_T$ is more likely incorrect. Right: The number of tokens used for the final prediction after the first reasoning step $t_1$, i.e., the number of tokens used for $[t_2, t_3, ..., t_T]$. Although $q_1$ is correct, the model still consumes a large amount of tokens for the following reasoning steps--overthinking.
• Figure 4: Majority voting accuracy with different number of samplings for four LLMs. The vertical dashed line denotes the smallest $M$ whose accuracy is equal to or larger than the accuracy of $N=64$.
• Figure 5: The pass rate on LiveCodeBench, where we set $N=16$ for early pruning (TopM).