SyncThink: A Training-Free Strategy to Align Inference Termination with Reasoning Saturation

TL;DR

Long-chain reasoning in CoT prompts improves problem solving but incurs substantial inference costs due to lengthy reasoning traces. SyncThink is a training-free decoding strategy that dynamically truncates reasoning by monitoring the logit rank of the reasoning-transition token </think> and its entropy, aligning decoding with information saturation. It formalizes the efficiency-accuracy trade-off, reveals a cognitive lag where accuracy saturates around $60\%$ progress, and demonstrates a strong, model-agnostic improvement in Pareto efficiency across GSM8K, MMLU, GPQA, and BBH on DeepSeek-R1 distills, achieving an average Top-1 of $62.00\%$ with roughly $70\%$ fewer tokens and up to $+8.1$ absolute accuracy on challenging tasks. The method is deployable and scalable across architectures, offering practical latency reductions while maintaining or improving solution quality.

Abstract

Chain-of-Thought (CoT) prompting improves reasoning but often produces long and redundant traces that substantially increase inference cost. We present SyncThink, a training-free and plug-and-play decoding method that reduces CoT overhead without modifying model weights. We find that answer tokens attend weakly to early reasoning and instead focus on the special token "/think", indicating an information bottleneck. Building on this observation, SyncThink monitors the model's own reasoning-transition signal and terminates reasoning. Experiments on GSM8K, MMLU, GPQA, and BBH across three DeepSeek-R1 distilled models show that SyncThink achieves 62.00 percent average Top-1 accuracy using 656 generated tokens and 28.68 s latency, compared to 61.22 percent, 2141 tokens, and 92.01 s for full CoT decoding. On long-horizon tasks such as GPQA, SyncThink can further yield up to +8.1 absolute accuracy by preventing over-thinking.

Figures (8)

• Figure 1: Efficiency-Accuracy Trade-off. The blue curve depicts the base model's capability boundary. SyncThink (red star) significantly surpasses this limit, occupying the top-left Pareto frontier with higher accuracy and reduced token usage. The bottom bars represent the marginal reasoning efficiency ($\Delta \text{Acc}/\Delta \text{Tokens}$). Our method achieves the highest score (1.08), demonstrating that it effectively retains critical reasoning while pruning redundancy. More results across four datasets can be viewed in Appendix \ref{['app:pareto_frontier_analysis']}.
• Figure 2: Empirical evidence of the Universal Reasoning Bottleneck on a GSM8K example. (a) At Layer 29, attention concentrates on the </think> boundary rather than reasoning tokens (more layers in Fig. \ref{['fig:attention_maps_appendix']}). (b) Saliency analysis shows that information transfer from Reasoning to Answer is negligible (red line), while the dominant pathway is mediated by </think> (purple line), suggesting reasoning is compressed into this transition token.
• Figure 3: The rationale behind SyncThink. (Top) Micro-Dynamics: The </think> rank provides a real-time signal that tracks the four reasoning phases in §\ref{['sec:rank_dynamics']}. (Bottom) Macro-Statistics: On BBH, we observe a cognitive lag: truncation accuracy (blue) saturates within the shaded optimal truncation zone well before the model’s intrinsic termination signal (red).
• Figure 4: Efficiency-Accuracy Trade-off. SyncThink demonstrates superior Pareto efficiency compared to Full CoT, SFT, and static truncation baselines. The red star indicates our method achieves the best balance between token reduction and model performance across all evaluated models.
• Figure 5: Ablation study on $\lambda$.