First-Step Advantage: Importance of Starting Right in Multi-Step Math Reasoning

TL;DR

First-Step Advantage investigates why small language models struggle to pick the correct starting point in multi-step reasoning and proposes QuestCoT, a self-questioning starting mechanism. The authors compare CoT, sub-question decomposition, and QuestCoT across GSM8K, SVAMP, ASDiv, and MultiArith, showing that guiding the initial step yields substantial performance gains for models in the 2–8B range. They demonstrate that the gains are not due to leakage of final answers and analyze why QuestCoT works, including its impact on unnecessary calculations, real-world knowledge, and context understanding. The work suggests a practical, cost-effective method to boost reasoning in smaller models, with broad implications for scalable reasoning-enabled systems and cautious considerations of limitations and biases.

Abstract

Language models can solve complex reasoning tasks better by learning to generate rationales for their predictions. Often these models know how to solve a task but their auto-regressive decoding nature leads to incorrect results if they start incorrectly. We observe that smaller models in particular when corrected, can solve a task that they would have otherwise struggled with. We demonstrate this phenomenon by using a larger model to guide smaller models, which leads to significantly improved performance (up to +24 points on the GSM8K dataset by 7B models). To assist smaller models in initiating the starting step, we propose QuestCoT, where a smaller model first asks itself how to start, before proceeding with a chain of reasoning. On various multistep mathematical reasoning datasets over multiple smaller models, we show that getting the right start can lead to significant performance gains across all models (gains of up to +6 points on GSM8K, +9 on SVAMP, +5 on ASDiv, and +7 on MultiArith).

Figures (11)

• Figure 1: Comparison between Chain-of-Thought (CoT) approach and QuestCoT. The CoT approach enables a Language Model (LM) to generate accurate answers through multiple samplings, yet it frequently struggles to confidently select the correct one. Conversely, QuestCoT utilizes self-question-guided generation, which facilitates the model's ability to choose the appropriate reasoning chain with higher confidence.
• Figure 2: Accuracy (if an answer exists in one of the output chains) comparison on GSM8K data set between different sized models: Mistral 7B, LLaMA-70B, and GPT-4.
• Figure 3: LLM-based first step guidance is highlighted in yellow followed by model generation.
• Figure 4: Example of a comparison between CoT reasoning and QuestCoT. QuestCoT first asks a question that helps to decide the first step and is highlighted in pink.
• Figure 5: Accuracy comparison between Subques and QuestCoT on the GSM8K and SVAMP datasets. Gemma refers to Gemma-2B, Phi3-Mini is Phi3-mini-3.8B, and LLaMA2, OlMo, and Mistral are all 7B variants, while LLaMA3 is LLaMA3-8B.