Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mitigating Misleading Chain-of-Thought Reasoning with Selective Filtering

Yexin Wu, Zhuosheng Zhang, Hai Zhao

TL;DR

The paper addresses unreliable chain-of-thought (CoT) reasoning in small language models by introducing SelF-Reasoner, a three-component system that generates a candidate reasoning chain, predicts the final answer, and critically filters CoT credibility via a CoT verifier. By applying CoT only when the reasoning chain is entailed by the question and confident, SelF-Reasoner improves accuracy over vanilla fine-tuning and two-level pipelines across ScienceQA, ECQA, and LastLetter, without relying on large LLMs at test time. The authors provide a detailed analysis of reasoning-chain quality, reveal common failure modes in CoT, and demonstrate a scalable “scaling law” for the CoT filter’s impact. The work offers a practical, interpretable approach to leveraging CoT in small models, with broad implications for robust reasoning and downstream applicability in resource-constrained settings.

Abstract

Large language models have manifested remarkable capabilities by leveraging chain-of-thought (CoT) reasoning techniques to solve intricate questions through step-by-step reasoning chains. Despite its success, the efficacy of such reasoning is inherently contingent upon the quality of CoT. However, flawless CoT reasoning cannot be guaranteed due to the presence of indecomposable questions and the potential for erroneous reasoning chains, particularly in the case of small-scale language models. To tackle this challenge, we propose a novel approach called the selective filtering reasoner (SelF-Reasoner) that assesses the entailment relationship between the question and the candidate reasoning chain. Then, we proceed with CoT reasoning when the reasoning chain demonstrates confidence; otherwise, we opt to predict the answer directly. SelF-Reasoner improves the fine-tuned T5 baseline consistently over the ScienceQA, ECQA, and LastLetter tasks. Code is available at \texttt{https://github.com/LibroWu/SelF-Reasoner}.

Mitigating Misleading Chain-of-Thought Reasoning with Selective Filtering

TL;DR

The paper addresses unreliable chain-of-thought (CoT) reasoning in small language models by introducing SelF-Reasoner, a three-component system that generates a candidate reasoning chain, predicts the final answer, and critically filters CoT credibility via a CoT verifier. By applying CoT only when the reasoning chain is entailed by the question and confident, SelF-Reasoner improves accuracy over vanilla fine-tuning and two-level pipelines across ScienceQA, ECQA, and LastLetter, without relying on large LLMs at test time. The authors provide a detailed analysis of reasoning-chain quality, reveal common failure modes in CoT, and demonstrate a scalable “scaling law” for the CoT filter’s impact. The work offers a practical, interpretable approach to leveraging CoT in small models, with broad implications for robust reasoning and downstream applicability in resource-constrained settings.

Abstract

Large language models have manifested remarkable capabilities by leveraging chain-of-thought (CoT) reasoning techniques to solve intricate questions through step-by-step reasoning chains. Despite its success, the efficacy of such reasoning is inherently contingent upon the quality of CoT. However, flawless CoT reasoning cannot be guaranteed due to the presence of indecomposable questions and the potential for erroneous reasoning chains, particularly in the case of small-scale language models. To tackle this challenge, we propose a novel approach called the selective filtering reasoner (SelF-Reasoner) that assesses the entailment relationship between the question and the candidate reasoning chain. Then, we proceed with CoT reasoning when the reasoning chain demonstrates confidence; otherwise, we opt to predict the answer directly. SelF-Reasoner improves the fine-tuned T5 baseline consistently over the ScienceQA, ECQA, and LastLetter tasks. Code is available at \texttt{https://github.com/LibroWu/SelF-Reasoner}.
Paper Structure (45 sections, 6 equations, 4 figures, 11 tables)

This paper contains 45 sections, 6 equations, 4 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Chain-of-thought Prompting
  4. Zero-Shot CoT.
  5. Few-Shot CoT.
  6. Fine-tuning Language Models to Reason
  7. Approach
  8. Baselines
  9. Vanilla Fine-tuning.
  10. Compound Generator.
  11. Two-level Pipeline.
  12. SelF-Reasoner
  13. Experiments
  14. Datasets
  15. Language Models
  16. ...and 30 more sections

Figures (4)

  • Figure 1: An example of an invalid CoT reasoning from ECQA aggarwal-etal-2021-explanationswang2023pinto. The errors are highlighted in red. The generated CoT is wrong at the first step, and the error continues to the end. However, when altering to direct prediction, this one-step reasoning question is solved correctly.
  • Figure 2: (a) Vanilla fine-tuning model that directly predicts the answer. (b) A compound generator that produces the reasoning chain and the answer simultaneously. (c) The two-level pipeline produces the reasoning chain first and then extracts the answer from it. (d) Our SelF-Reasoner. The Reasoner is fine-tuned to generate a solution based on the input question description. The Answerer is the model that can predict the answer directly or extract the answer from a question-solution pair. CoT Filter can determine whether the solution generated can be passed to the Answerer. Here, Q stands for Question, R stands for Reasoning Chain, and A stands for Answer. The symbol QR means concatenating Q and R, RA means concatenating R and A.
  • Figure 3: The "scaling law" in the size of the CoT filter and pipeline on ScienceQA benchmark. The dashed line presents the accuracy of the pipeline.
  • Figure 4: An example that does not conform to the CoT format and an expected reasoning chain. Blue texts are the knowledge lectures and Orange texts are the solutions.