Strategic Chain-of-Thought: Guiding Accurate Reasoning in LLMs through Strategy Elicitation

TL;DR

<3-5 sentence high-level summary> Strategic Chain-of-Thought (SCoT) addresses the instability of standard Chain-of-Thought prompts by first eliciting a problem-solving strategy and then using that strategy to guide intermediate reasoning and final answers within a single prompt. The approach extends to a few-shot setting via a strategy-driven demonstration corpus and demonstration matching, enabling stronger performance across eight reasoning datasets with various models. Empirical results show substantial gains on GSM8K and Tracking_Object, among others, and analyses demonstrate component contributions, efficiency trade-offs, and robustness across model scales. Overall, SCoT offers a practical, resource-efficient path to more reliable and accurate reasoning in large language models.

Abstract

The Chain-of-Thought (CoT) paradigm has emerged as a critical approach for enhancing the reasoning capabilities of large language models (LLMs). However, despite their widespread adoption and success, CoT methods often exhibit instability due to their inability to consistently ensure the quality of generated reasoning paths, leading to sub-optimal reasoning performance. To address this challenge, we propose the \textbf{Strategic Chain-of-Thought} (SCoT), a novel methodology designed to refine LLM performance by integrating strategic knowledge prior to generating intermediate reasoning steps. SCoT employs a two-stage approach within a single prompt: first eliciting an effective problem-solving strategy, which is then used to guide the generation of high-quality CoT paths and final answers. Our experiments across eight challenging reasoning datasets demonstrate significant improvements, including a 21.05\% increase on the GSM8K dataset and 24.13\% on the Tracking\_Objects dataset, respectively, using the Llama3-8b model. Additionally, we extend the SCoT framework to develop a few-shot method with automatically matched demonstrations, yielding even stronger results. These findings underscore the efficacy of SCoT, highlighting its potential to substantially enhance LLM performance in complex reasoning tasks.

Figures (16)

• Figure 1: Comparison of some popular methods with SCoT: As a single-query method, SCoT is efficient and does not rely on external knowledge sources, distinguishing it from other approaches.
• Figure 2: Illustration of Zero-shot and Few-shot Strategic SCoT. Few-shot SCoT builds upon Zero-shot SCoT by incorporating selected demonstrations. Details of the Few-shot SCoT approach are omitted due to space limitations.
• Figure 3: Prompt templates for zero-shot and few-shot SCoT
• Figure 4: Example of a Workflow in a Math Task Prompt
• Figure 5: Accuracy(%) across three datasets using different scales of models in Llama2 series