Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Theoretical Understanding of Chain-of-Thought: Coherent Reasoning and Error-Aware Demonstration

Yingqian Cui, Pengfei He, Xianfeng Tang, Qi He, Chen Luo, Jiliang Tang, Yue Xing

TL;DR

The paper develops a theoretical framework comparing Coherent CoT with Stepwise ICL, showing that treating CoT as a holistic process improves inference and error correction. It derives closed-form insights under a two-layer noisy regression model and demonstrates that Coherent CoT minimizes prediction loss more effectively than Stepwise ICL, while revealing heightened sensitivity to intermediate-step perturbations. Building on this, the authors propose error-aware demonstrations that include both correct and incorrect reasoning paths, aiming to fortify intermediate-step accuracy and overall performance. Experimental results across multiple LLMs and benchmarks corroborate the benefits of the proposed approach, with notable improvements when error explanations accompany incorrect reasoning paths. The work highlights the practical significance of considering entire reasoning chains during training and the value of explicit error-focused demonstrations for reliable CoT behavior.

Abstract

Few-shot Chain-of-Thought (CoT) prompting has demonstrated strong performance in improving the reasoning capabilities of large language models (LLMs). While theoretical investigations have been conducted to understand CoT, the underlying transformer used in these studies isolates the CoT reasoning process into separated in-context learning steps (Stepwise ICL). In this work, we theoretically show that, compared to Stepwise ICL, the transformer gains better error correction ability and more accurate predictions if the reasoning from earlier steps (Coherent CoT) is integrated. Given that this coherent reasoning changes the behavior of the transformer, we further investigate the sensitivity of the transformer with Coherent CoT when the demonstration examples are corrupted at the inference stage. Our theoretical results indicate that the transformer is more sensitive to errors in intermediate reasoning steps than the final outcome. Building upon this observation, we propose an improvement on CoT by incorporating both correct and incorrect reasoning paths in the demonstration. Our experiments validate the effectiveness of the proposed approach.

A Theoretical Understanding of Chain-of-Thought: Coherent Reasoning and Error-Aware Demonstration

TL;DR

The paper develops a theoretical framework comparing Coherent CoT with Stepwise ICL, showing that treating CoT as a holistic process improves inference and error correction. It derives closed-form insights under a two-layer noisy regression model and demonstrates that Coherent CoT minimizes prediction loss more effectively than Stepwise ICL, while revealing heightened sensitivity to intermediate-step perturbations. Building on this, the authors propose error-aware demonstrations that include both correct and incorrect reasoning paths, aiming to fortify intermediate-step accuracy and overall performance. Experimental results across multiple LLMs and benchmarks corroborate the benefits of the proposed approach, with notable improvements when error explanations accompany incorrect reasoning paths. The work highlights the practical significance of considering entire reasoning chains during training and the value of explicit error-focused demonstrations for reliable CoT behavior.

Abstract

Few-shot Chain-of-Thought (CoT) prompting has demonstrated strong performance in improving the reasoning capabilities of large language models (LLMs). While theoretical investigations have been conducted to understand CoT, the underlying transformer used in these studies isolates the CoT reasoning process into separated in-context learning steps (Stepwise ICL). In this work, we theoretically show that, compared to Stepwise ICL, the transformer gains better error correction ability and more accurate predictions if the reasoning from earlier steps (Coherent CoT) is integrated. Given that this coherent reasoning changes the behavior of the transformer, we further investigate the sensitivity of the transformer with Coherent CoT when the demonstration examples are corrupted at the inference stage. Our theoretical results indicate that the transformer is more sensitive to errors in intermediate reasoning steps than the final outcome. Building upon this observation, we propose an improvement on CoT by incorporating both correct and incorrect reasoning paths in the demonstration. Our experiments validate the effectiveness of the proposed approach.

Paper Structure

This paper contains 37 sections, 8 theorems, 48 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Empirical Insights in CoT.
  4. Theories in ICL and CoT.
  5. Theoretical Results
  6. Model Setup
  7. Data generation process.
  8. Coherent CoT and Stepwise ICL.
  9. Model architecture.
  10. Training objectives.
  11. Coherent CoT vs Stepwise ICL
  12. Sensitivity against Random Perturbation
  13. Methodology
  14. Experiments
  15. Experimental setup
  16. ...and 22 more sections

Key Result

Theorem 3.1

Under Assumption ass:gen and ass:opt, the optimal expected loss of Stepwise ICL is achieved when $u_x \neq 0$ and $u_z \neq 0$ are satisfied. The corresponding optimal loss is

Figures (3)

  • Figure 1: Comparison of Coherent CoT and Stepwise ICL
  • Figure 2: CoT's sensitivity to different steps (The green and red lines overlap because the results for adding noise to $y$ and the case of no noise are very close to each other.)
  • Figure 3: Standard CoT prompting vs the proposed method: CoT prompting with correct & incorrect paths.

Theorems & Definitions (12)

  • Definition 3.1: Stepwise ICL
  • Definition 3.2: Coherent CoT
  • Theorem 3.1
  • Remark 3.1
  • Theorem 3.2
  • Proposition 3.1
  • Proposition 3.2
  • Theorem 3.3
  • Theorem 3.4
  • Theorem 3.5
  • ...and 2 more