The Zero-Step Thinking: An Empirical Study of Mode Selection as Harder Early Exit in Reasoning Models

TL;DR

This paper formalizes Mode Selection as a harder variant of Early Exit, requiring a decision on the reasoning mode at zero-step before any explicit thinking. It systematically compares prompt-based and internal-state methods across multiple datasets and model scales, finding that prompt-based approaches struggle with minimal information while internal-state methods generally perform better but suffer from stability. The study demonstrates that existing evaluation metrics (e.g., ROC-AUC, ECE, Brier) do not fully explain the observed behavior, underscoring the need for deeper analysis of model internals in adaptive reasoning. Overall, the work highlights the potential and challenges of Zero-Step Thinking and NoThinking for efficient yet reliable adaptive reasoning in large language models, and provides a foundation for developing more robust mode-selection strategies.

Abstract

Reasoning models have demonstrated exceptional performance in tasks such as mathematics and logical reasoning, primarily due to their ability to engage in step-by-step thinking during the reasoning process. However, this often leads to overthinking, resulting in unnecessary computational overhead. To address this issue, Mode Selection aims to automatically decide between Long-CoT (Chain-of-Thought) or Short-CoT by utilizing either a Thinking or NoThinking mode. Simultaneously, Early Exit determines the optimal stopping point during the iterative reasoning process. Both methods seek to reduce the computational burden. In this paper, we first identify Mode Selection as a more challenging variant of the Early Exit problem, as they share similar objectives but differ in decision timing. While Early Exit focuses on determining the best stopping point for concise reasoning at inference time, Mode Selection must make this decision at the beginning of the reasoning process, relying on pre-defined fake thoughts without engaging in an explicit reasoning process, referred to as zero-step thinking. Through empirical studies on nine baselines, we observe that prompt-based approaches often fail due to their limited classification capabilities when provided with minimal hand-crafted information. In contrast, approaches that leverage internal information generally perform better across most scenarios but still exhibit issues with stability. Our findings indicate that existing methods relying solely on the information provided by models are insufficient for effectively addressing Mode Selection in scenarios with limited information, highlighting the ongoing challenges of this task. Our code is available at https://github.com/Trae1ounG/Zero_Step_Thinking.

Figures (3)

• Figure 1: Illustration of Early Exit and Mode Selection. Early Exit employs an iterative monitor to decide whether to stop reasoning at the end of each step in the thought process. In contrast, Mode Selection operates prior to explicit reasoning, determining the optimal thinking strategy at zero-step. While both methods can leverage the same monitoring mechanisms, they differ in their timing: Early Exit monitors iteratively during the reasoning process, whereas Mode Selection monitors only once at the beginning to append </think> and terminate the reasoning process.
• Figure 2: Trade-off between average accuracy and average output tokens across different methods and three model sizes. Each point on the curve represents a distinct $\lambda$ value from 0.1 to 1.0.
• Figure 3: ROC-AUC comparison between different model sizes: (a) 1.5B and (b) 7B.