What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoning

TL;DR

This work addresses the question of what structural features in long chain-of-thought reasoning predict correct outcomes in large language models. It introduces LCoT2Tree, an automated pipeline that converts sequential LCoTs into hierarchical trees and analyzes them with graph neural networks to extract patterns such as exploration, backtracking, and verification. The study demonstrates that these structural patterns are stronger predictors of reasoning success than simple length-based metrics, provides explainability through identifying influential subgraphs, and reveals task- and model-specific differences in reasoning behavior. Practically, integrating the tree-based reasoning signals into Best-of-N decoding improves selection quality across diverse tasks and models. Overall, the work positions internal reasoning structure as a critical diagnostic and optimization target for improving LLM reasoning performance.

Abstract

Recent advances in reasoning with large language models (LLMs) have popularized Long Chain-of-Thought (LCoT), a strategy that encourages deliberate and step-by-step reasoning before producing a final answer. While LCoTs have enabled expert-level performance in complex tasks, how the internal structures of their reasoning chains drive, or even predict, the correctness of final answers remains a critical yet underexplored question. In this work, we present LCoT2Tree, an automated framework that converts sequential LCoTs into hierarchical tree structures and thus enables deeper structural analysis of LLM reasoning. Using graph neural networks (GNNs), we reveal that structural patterns extracted by LCoT2Tree, including exploration, backtracking, and verification, serve as stronger predictors of final performance across a wide range of tasks and models. Leveraging an explainability technique, we further identify critical thought patterns such as over-branching that account for failures. Beyond diagnostic insights, the structural patterns by LCoT2Tree support practical applications, including improving Best-of-N decoding effectiveness. Overall, our results underscore the critical role of internal structures of reasoning chains, positioning LCoT2Tree as a powerful tool for diagnosing, interpreting, and improving reasoning in LLMs.

Figures (17)

• Figure 1: The distribution of output token length for correctly answered (Positive) and incorrectly answered (Negative) samples by DeepSeek-R1-Distill-Qwen-32B on two datasets.
• Figure 2: Data count and accuracy of the MATH dataset for DeepSeek-32B across varying response lengths. Accuracy notably declines as response length increases.
• Figure 3: The workflow for LCoT2Tree. It transforms sequential long chain-of-thought into reasoning tree, which involves five steps: (1) Extract Sketch, (2) Split Thought, (3) Assign Step, (4) Identify Function, and (5) Build Tree.
• Figure 4: Visualization and frequency of four structural error patterns across three datasets. (A) Over Branching: abundance of explorations or verifications within a single node, (B) Step Redundancy: over-generation of thoughts within a single reasoning step, (C) Direct Reasoning: following a straight, minimal-branch path from one step to a much deeper step, and (D) Skipped Thinking: jumping multiple steps ahead without intermediate logical analysis. Representative example of each class is available in Figure \ref{['appendix:visual_error_case']}.
• Figure 5: Accuracy comparison of different Best-of-N decoding strategies on the LCB-v6 benchmark.