Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Check: Unleashing Potentials for Self-Correction in Large Language Models

Che Zhang, Zhenyang Xiao, Chengcheng Han, Yixin Lian, Yuejian Fang

TL;DR

The paper tackles improving self-correction in large language models by focusing on self-checking capabilities for reasoning tasks. It introduces Step CoT Check, a fine-grained, step-by-step checking format that categorizes errors and guides correction, and builds a checking-correction dataset to train LLMs. Empirical results show that finetuning with Step CoT Check substantially enhances self-checking and self-correction, with larger models deriving greater benefits. The work provides insights into data composition, prompting strategies, and the cost-performance trade-offs, and releases resources to support reproducibility and further research.

Abstract

Self-correction has achieved impressive results in enhancing the style and security of the generated output from large language models (LLMs). However, recent studies suggest that self-correction might be limited or even counterproductive in reasoning tasks due to LLMs' difficulties in identifying logical mistakes. In this paper, we aim to enhance the self-checking capabilities of LLMs by constructing training data for checking tasks. Specifically, we apply the Chain of Thought (CoT) methodology to self-checking tasks, utilizing fine-grained step-level analyses and explanations to assess the correctness of reasoning paths. We propose a specialized checking format called "Step CoT Check". Following this format, we construct a checking-correction dataset that includes detailed step-by-step analysis and checking. Then we fine-tune LLMs to enhance their error detection and correction abilities. Our experiments demonstrate that fine-tuning with the "Step CoT Check" format significantly improves the self-checking and self-correction abilities of LLMs across multiple benchmarks. This approach outperforms other formats, especially in locating the incorrect position, with greater benefits observed in larger models. For reproducibility, all the datasets and code are provided in https://github.com/bammt/Learn-to-check.

Learning to Check: Unleashing Potentials for Self-Correction in Large Language Models

TL;DR

The paper tackles improving self-correction in large language models by focusing on self-checking capabilities for reasoning tasks. It introduces Step CoT Check, a fine-grained, step-by-step checking format that categorizes errors and guides correction, and builds a checking-correction dataset to train LLMs. Empirical results show that finetuning with Step CoT Check substantially enhances self-checking and self-correction, with larger models deriving greater benefits. The work provides insights into data composition, prompting strategies, and the cost-performance trade-offs, and releases resources to support reproducibility and further research.

Abstract

Self-correction has achieved impressive results in enhancing the style and security of the generated output from large language models (LLMs). However, recent studies suggest that self-correction might be limited or even counterproductive in reasoning tasks due to LLMs' difficulties in identifying logical mistakes. In this paper, we aim to enhance the self-checking capabilities of LLMs by constructing training data for checking tasks. Specifically, we apply the Chain of Thought (CoT) methodology to self-checking tasks, utilizing fine-grained step-level analyses and explanations to assess the correctness of reasoning paths. We propose a specialized checking format called "Step CoT Check". Following this format, we construct a checking-correction dataset that includes detailed step-by-step analysis and checking. Then we fine-tune LLMs to enhance their error detection and correction abilities. Our experiments demonstrate that fine-tuning with the "Step CoT Check" format significantly improves the self-checking and self-correction abilities of LLMs across multiple benchmarks. This approach outperforms other formats, especially in locating the incorrect position, with greater benefits observed in larger models. For reproducibility, all the datasets and code are provided in https://github.com/bammt/Learn-to-check.
Paper Structure (27 sections, 4 figures, 4 tables)

This paper contains 27 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Checking-Correction Data Generation
  5. Collecting Reasoning Paths with Errors
  6. Error Type Definition
  7. Checking Data Generation
  8. Correction Data Generation
  9. Fine-Tuning LLMs
  10. Experiments
  11. Experiment Step
  12. CoT Dataset
  13. Generate the Checking-Correction Dataset
  14. Fine-Tuning and Evaluation
  15. Main Results and Analysis
  16. ...and 12 more sections

Figures (4)

  • Figure 1: The process of checking-correction dataset generation.
  • Figure 2: The reasoning process combined with self-correction. The Process involves three stages. Stage 1 involves direct reasoning for a given question. Stage 2 examines the direct reasoning path and provides feedback at the end. If Stage 2 check identifies errors, Stage 3 is then initiated. Stage 3 requires the model to generate a new reasoning path and attempt to correct the erroneous answer given the question, original answer, and feedback.
  • Figure 3: Accuracy($\%$) of checking the correctness of reasoning paths obtained by itself.
  • Figure 4: Accuracy ($\%$) of direct reasoning and reasoning combined with self-correction on GSM8K as checking-corection data types increase.