Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns

TL;DR

This work tackles noise and interpretability issues in unsupervised chain-of-thought prompting by introducing Pattern-CoT, which selects demonstrations based on diverse reasoning patterns rather than semantic similarity alone. The method collects seed rationales, discovers task-relevant reasoning operations, clusters pattern representations with k-means, and selects representative demonstrations from each cluster, guided by a knowledge-informed rule for the number of demonstrations $k = \left\lceil \tfrac{1}{2} \times n \times \left(1 + \log(N)\right) \right\rceil$. Across eight reasoning datasets and multiple open-source LLMs, Pattern-CoT delivers robust improvements, with case studies and attribution analyses revealing clearer and more accurate reasoning paths. The approach enhances interpretability and demonstrates strong cross-model applicability, offering a scalable and noise-resistant alternative to semantic-based CoT prompting with practical impact for reasoning tasks in AI systems.

Abstract

Chain of Thought (CoT) prompting can encourage language models to engage in multi-step logical reasoning. The quality of the provided demonstrations significantly influences the success of downstream inference tasks. Current unsupervised CoT methods primarily select examples based on the semantics of the questions, which can introduce noise and lack interpretability. In this paper, we propose leveraging reasoning patterns to enhance CoT prompting effectiveness. Reasoning patterns represent the process by which language models arrive at their final results. By utilizing prior knowledge and prompt-based methods from large models, we first construct task-specific pattern sets. We then select diverse demonstrations based on different reasoning patterns. This approach not only mitigates the impact of noise but also provides explicit interpretability to help us understand the mechanisms of CoT. Extensive experiments demonstrate that our method is more robust and consistently leads to improvements across various reasoning tasks.

Figures (4)

• Figure 1: Example of the chain-of-thought prompting. The prompt influences how LLMs arrive at the final answer.
• Figure 2: Illustration of our proposed framework. We first extract different patterns from the original rationales. Then clustering is used to produce a group of demonstrations. This enables LLMs to perceive diverse reasoning patterns and to select a proper solution path. It avoids LLMs being biased by monotonous reasoning mode.
• Figure 3: Comparison of different operation sets.
• Figure 4: Visualization of token attribution for the case study. The left part stands for the score matrix of patterns from Auto-CoT, and the right part stands for the score matrix from our method. The upper column denotes each individual prompt, and the row denotes the generated token sequence. Higher scores (positive) indicate that the input has a greater impact on the output.