Do LLMs Really Think Step-by-step In Implicit Reasoning?
Yijiong Yu
TL;DR
The paper probes whether implicit chain-of-thought (CoT) reasoning—either triggered by prompts or learned through training—truly imitates explicit CoT in multi-step arithmetic tasks. By training a linear probe on hidden-layer representations, it reveals that prompted implicit CoT rarely encodes intermediate steps, while internally trained CoT shows stepwise computation, though with limited capacity and sensitivity to problem formatting. The findings indicate implicit CoT cannot yet replace explicit CoT for reliability and robustness, particularly under format changes, reinforcing the continued practicality of explicit CoT for demanding reasoning tasks. The work highlights a latency-accuracy trade-off and underscores the need for robust, format-insensitive reasoning strategies in LLMs.
Abstract
It has been well-known that Chain-of-Thought can remarkably enhance LLMs' performance on complex tasks. However, because it also introduces slower inference speeds and higher computational costs, many researches have attempted to use implicit CoT, which does not need LLMs to explicitly generate the intermediate steps. However, the invisible reasoning process leaves us a doubt that, can implicit CoT really be equal to explicit CoT? Therefore, in this study, we address this question through experiments. We probe the information of intermediate steps from the model's hidden states when it is either trained or prompted to perform implicit CoT. The results surprisingly indicate that when prompted, LLMs hardly think about intermediate steps, suggesting they may just rely on experience rather than strict step-by-step reasoning. But when trained, they indeed calculate intermediate steps. Moreover, in both situations, we find the effect of using implicit CoT is susceptible to the format of the problem, reaffirming the current deficiency of implicit CoT.