Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Models

TL;DR

The paper tackles how reasoning in large language models evolves during training by examining internal representations rather than solely external outputs. Using a dual framework that combines probing-based representation quality (including V-probing) and generation accuracy across tasks, it reveals that post-training yields limited gains in static initial representations but markedly improves the model’s ability to steer dynamic representations through chain-of-thought generation. Statistical analyses show final representations correlate more with correct generation than initial ones, while counterfactuals demonstrate that the semantic content of CoT, not mere computation or parameter changes, drives the critical state transitions. These findings offer a representational lens for reasoning interpretability and inform future optimization and self-training strategies targeting internal states to enhance reasoning efficiency and reliability.

Abstract

Large Language Models have achieved remarkable performance on reasoning tasks, motivating research into how this ability evolves during training. Prior work has primarily analyzed this evolution via explicit generation outcomes, treating the reasoning process as a black box and obscuring internal changes. To address this opacity, we introduce a representational perspective to investigate the dynamics of the model's internal states. Through comprehensive experiments across models at various training stages, we discover that post-training yields only limited improvement in static initial representation quality. Furthermore, we reveal that, distinct from non-reasoning tasks, reasoning involves a significant continuous distributional shift in representations during generation. Comparative analysis indicates that post-training empowers models to drive this transition toward a better distribution for task solving. To clarify the relationship between internal states and external outputs, statistical analysis confirms a high correlation between generation correctness and the final representations; while counterfactual experiments identify the semantics of the generated tokens, rather than additional computation during inference or intrinsic parameter differences, as the dominant driver of the transition. Collectively, we offer a novel understanding of the reasoning process and the effect of training on reasoning enhancement, providing valuable insights for future model analysis and optimization.

Figures (27)

• Figure 1: Overview of the dual-perspective analysis framework. We conduct a comprehensive study across four representative tasks, measuring representation quality via probing alongside explicit generation accuracy. Our results indicate that post-training significantly improves generation accuracy but yields limited enhancement to static initial representation quality. Furthermore, tracking representation dynamics reveals that the reasoning process involves a significant distributional shift, where post-training empowers models to drive representations toward a higher-quality state. Finally, we investigate the relationship between explicit generation and implicit representation, quantifying their alignment and identifying the dominant factor driving this state transition.
• Figure 2: Training relationships among the Qwen2.5-7B series.
• Figure 3: Development of initial representation quality and generation accuracy. The gray dashed lines indicate the baseline performance of the Base model for probing and generation. See Appendix \ref{['sec:exp1 more']} for full results across all tasks.
• Figure 4: Representation quality dynamics during generation. Trends are analyzed using linear regression and Spearman rank correlation. We mark the final probing accuracy of the strong reasoning models with ★. See Appendix \ref{['sec:exp2 more']} for full results.
• Figure 5: Representation quality remains relatively stable during generation on the factuality task with minimal reasoning.

Theorems & Definitions (1)

• Proposition 3.1: Capacity-Constrained Accuracy Bound