EntroCoT: Enhancing Chain-of-Thought via Adaptive Entropy-Guided Segmentation

TL;DR

EntroCoT addresses the problem of correct final answers being supported by flawed intermediate reasoning in chain-of-thought datasets. It introduces an entropy-guided segmentation of CoT traces at high-uncertainty tokens and uses Monte Carlo rollouts to verify that each segment monotonically improves the likelihood of the correct answer, retaining only reliable traces for distillation. The approach yields consistent accuracy gains across six mathematics benchmarks and two base models, even while filtering out a substantial portion of the data, and demonstrates the importance of carefully segmented, behaviorally validated reasoning over simply increasing data volume. The method is model- and domain-agnostic within tasks that have verifiable final answers, offering a scalable path to higher-fidelity CoT data and improved generalization in mathematical reasoning systems.

Abstract

Chain-of-Thought (CoT) prompting has significantly enhanced the mathematical reasoning capabilities of Large Language Models. We find existing fine-tuning datasets frequently suffer from the "answer right but reasoning wrong" probelm, where correct final answers are derived from hallucinated, redundant, or logically invalid intermediate steps. This paper proposes EntroCoT, a unified framework for automatically identifying and refining low-quality CoT supervision traces. EntroCoT first proposes an entropy-based mechanism to segment the reasoning trace into multiple steps at uncertain junctures, and then introduces a Monte Carlo rollout-based mechanism to evaluate the marginal contribution of each step. By accurately filtering deceptive reasoning samples, EntroCoT constructs a high-quality dataset where every intermediate step in each reasoning trace facilitates the final answer. Extensive experiments on mathematical benchmarks demonstrate that fine-tuning on the subset constructed by EntroCoT consistently outperforms the baseslines of full-dataset supervision.

Figures (4)

• Figure 1: In chain-of-thought reasoning, high-entropy tokens function as forks introducing alternative reasoning branches, whereas low-entropy tokens proceed along the established path.
• Figure 2: Token-level entropy is computed along the CoT reasoning path, and the sequence is evenly divided into early, middle, and late segments by token order. Based on the distribution of high-entropy positions within each segment, a greedy algorithm selects spatially dispersed high-entropy positions as segmentation points (red dots) for subsequent construction of multiple prompts.
• Figure 3: Token-level entropy is computed over the CoT, and high-entropy positions are analyzed across early, middle, and late segments. Segmentation points are adaptively selected via greedy search to match the segment-wise entropy distribution, partitioning the CoT into sub-segments. The question combined with progressively accumulated CoT segments is used to prompt a lightweight model for multiple rollouts; the last segment is excluded to avoid answer leakage. Samples whose CoT segments' accuracy remains non-decreasing are retained as reliable data.
• Figure 4: Dataset sizes for MetaMathQA and NuminaMath under different training strategies. Direct SFT denotes standard supervised fine-tuning, while EntroCoT variants differ in sampling and greedy constraints.