Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decomposing LLM Self-Correction: The Accuracy-Correction Paradox and Error Depth Hypothesis

Yin Li

TL;DR

The paper presents a Self-Correction Decomposition framework that splits LLM self-improvement into error detection, localization, and correction. Through cross-model experiments on GSM8K-Complex using three architectures, it reveals the Accuracy-Correction paradox: stronger models exhibit lower intrinsic correction rates than weaker ones. It introduces the Error Depth Hypothesis, arguing that deeper, more fundamental errors in stronger models resist self-correction while shallower errors in weaker models are more tractable, and shows that error-detection capability is highly architecture-dependent. The findings have practical implications for designing self-refinement pipelines, suggesting that iterative reflection can compensate for weak detection and that model-generated hints may hurt, especially in high-stakes reasoning tasks. The work also notes limitations and proposes directions for scaling, improved localization, and cross-domain validation.

Abstract

Large Language Models (LLMs) are widely believed to possess self-correction capabilities, yet recent studies suggest that intrinsic self-correction--where models correct their own outputs without external feedback--remains largely ineffective. In this work, we systematically decompose self-correction into three distinct sub-capabilities: error detection, error localization, and error correction. Through cross-model experiments on GSM8K-Complex (n=500 per model, 346 total errors) with three major LLMs, we uncover a striking Accuracy-Correction Paradox: weaker models (GPT-3.5, 66% accuracy) achieve 1.6x higher intrinsic correction rates than stronger models (DeepSeek, 94% accuracy)--26.8% vs 16.7%. We propose the Error Depth Hypothesis: stronger models make fewer but deeper errors that resist self-correction. Error detection rates vary dramatically across architectures (10% to 82%), yet detection capability does not predict correction success--Claude detects only 10% of errors but corrects 29% intrinsically. Surprisingly, providing error location hints hurts all models. Our findings challenge linear assumptions about model capability and self-improvement, with important implications for the design of self-refinement pipelines.

Decomposing LLM Self-Correction: The Accuracy-Correction Paradox and Error Depth Hypothesis

TL;DR

The paper presents a Self-Correction Decomposition framework that splits LLM self-improvement into error detection, localization, and correction. Through cross-model experiments on GSM8K-Complex using three architectures, it reveals the Accuracy-Correction paradox: stronger models exhibit lower intrinsic correction rates than weaker ones. It introduces the Error Depth Hypothesis, arguing that deeper, more fundamental errors in stronger models resist self-correction while shallower errors in weaker models are more tractable, and shows that error-detection capability is highly architecture-dependent. The findings have practical implications for designing self-refinement pipelines, suggesting that iterative reflection can compensate for weak detection and that model-generated hints may hurt, especially in high-stakes reasoning tasks. The work also notes limitations and proposes directions for scaling, improved localization, and cross-domain validation.

Abstract

Large Language Models (LLMs) are widely believed to possess self-correction capabilities, yet recent studies suggest that intrinsic self-correction--where models correct their own outputs without external feedback--remains largely ineffective. In this work, we systematically decompose self-correction into three distinct sub-capabilities: error detection, error localization, and error correction. Through cross-model experiments on GSM8K-Complex (n=500 per model, 346 total errors) with three major LLMs, we uncover a striking Accuracy-Correction Paradox: weaker models (GPT-3.5, 66% accuracy) achieve 1.6x higher intrinsic correction rates than stronger models (DeepSeek, 94% accuracy)--26.8% vs 16.7%. We propose the Error Depth Hypothesis: stronger models make fewer but deeper errors that resist self-correction. Error detection rates vary dramatically across architectures (10% to 82%), yet detection capability does not predict correction success--Claude detects only 10% of errors but corrects 29% intrinsically. Surprisingly, providing error location hints hurts all models. Our findings challenge linear assumptions about model capability and self-improvement, with important implications for the design of self-refinement pipelines.
Paper Structure (39 sections, 3 figures, 3 tables)

This paper contains 39 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Self-Refinement in LLMs.
  4. Error Detection and Verification.
  5. Mathematical Reasoning.
  6. Methodology
  7. Problem Formulation
  8. Experimental Setup
  9. Dataset.
  10. GSM8K-Complex (Ours).
  11. Models.
  12. Evaluation Protocol.
  13. Metrics
  14. Results
  15. Cross-Model Comparison
  16. ...and 24 more sections

Figures (3)

  • Figure 1: The Self-Correction Decomposition Framework. We separate self-correction into three distinct, independently measurable capabilities.
  • Figure 2: The Accuracy-Correction Paradox. The strongest model (DeepSeek, 94% accuracy) achieves the lowest intrinsic correction rate (16.7%), while weaker models correct 1.6--1.7$\times$ more errors.
  • Figure 3: Iterative Reflection Dynamics. DeepSeek saturates after round 1 (20%), while GPT-3.5 and Claude continue improving through round 3 (68%, 61%).