Critical Thinking: Which Kinds of Complexity Govern Optimal Reasoning Length?

TL;DR

This work introduces a DFA-based framework to study how task structure governs the optimal amount of reasoning tokens (the critical length $L^*$) used by LLMs at inference. It demonstrates a consistent existence of an $L^*$ that maximizes accuracy across tasks and models, and shows that $L^*$ correlates strongly with the DFA run length $N$ but weakly with the DFA state-space size $k$, implying that reasoning length primarily supports latent state tracking rather than representing larger state spaces. The authors show that a simple predictor using $(k,N)$ can forecast $L^*$ with $R^2 \,\approx\,0.65$ and that filtering generations to near the predicted $L^*$ yields measurable accuracy gains, especially for larger or COT-RL models. These findings offer actionable guidance for inference-time compute: tailoring reasoning length to task structure can improve accuracy and efficiency, motivating future work on more refined predictors and extensions to complex tasks.

Abstract

Large language models (LLMs) often benefit from verbalized reasoning at inference time, but it remains unclear which aspects of task difficulty these extra reasoning tokens address. To investigate this question, we formalize a framework using deterministic finite automata (DFAs). DFAs offer a formalism through which we can characterize task complexity through measurable properties such as run length (number of reasoning steps required) and state-space size (decision complexity). We first show that across different tasks and models of different sizes and training paradigms, there exists an optimal amount of reasoning tokens such that the probability of producing a correct solution is maximized. We then investigate which properties of complexity govern this critical length: we find that task instances with longer corresponding underlying DFA runs (i.e. demand greater latent state-tracking requirements) correlate with longer reasoning lengths, but, surprisingly, that DFA size (i.e. state-space complexity) does not. We then demonstrate an implication of these findings: being able to predict the optimal number of reasoning tokens for new problems and filtering out non-optimal length answers results in consistent accuracy improvements.

Figures (9)

• Figure 1: Average per-task accuracy as a function of how many tokens a model includes in its response. Across tasks and models, accuracy peaks at a critical length ($L^*$). Each line represents a different model: COT-RL lines in orange, regular Instruction-tuned models in blue, with darker hues indicating larger models. Sequence length is normalized so that 0 represents the least tokens a model produces per task while $L^*$ is the peak.
• Figure 1: The large language models we use in our analysis. The last column refers to the model's final RL training stage: COT-RL or Instruction Tuning.
• Figure 2: The distribution of generation lengths per model and corresponding critical lengths, averaged across tasks. COT-RL models (orange) generate longer sequences and have higher critical lengths.
• Figure 3: We systematically vary the complexity of test examples by formulating tasks as DFAs. The left panel of this figure shows a demonstrative parity task with a small run length ($N=4$) and state space ($k=2$). In the middle panel, we increase the run length while keeping state space size constant ($N=5$, $k=2$). In the final panel, we keep the run length from the first panel and increase state space size ($N=4$, $k=3$).
• Figure 4: On the left, critical length ($L^*$) is plotted as the run length and number of states are varied. Each line represents a different model: blue lines indicate standard instruction-tuned models, orange lines indicate COT-RL models, and darker hues indicate larger models. On the right, the correlation between $L^*$ and run length $(N)$ as well as number of states $(k)$. The values of $L^*$ are normalized per model and task for both figures. The critical length correlates strongly with run length $N$, but weakly with state-space size $k$.