DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inference

TL;DR

This work identifies a counterintuitive overthinking pattern in reasoning LLMs, evidenced by a three-region, U-shaped entropy curve across problem difficulty. It introduces DiffAdapt, a lightweight, training-free framework that uses a hidden-state probe to predict problem difficulty and adaptively selects Easy/Normal/Hard inference strategies for each question, reducing token usage while maintaining or improving accuracy. Oracle analyses show substantial efficiency and accuracy gains, and experiments across five models and eight benchmarks demonstrate robust performance under in-domain and out-of-domain conditions, with orthogonality to Length Control RL methods. The approach enables compute-efficient, high-quality reasoning without fine-tuning LLMs and is compatible with standard inference optimizations, offering practical impact for scalable reasoning systems.

Abstract

Recent reasoning Large Language Models (LLMs) demonstrate remarkable problem-solving abilities but often generate long thinking traces whose utility is unclear. Our work aims to improve their efficiency, enabling them to reach high performance without overthinking. First, we analyze the entropy of token probabilities in reasoning traces. Across three models, we observe a consistent U-shaped entropy pattern: high entropy on easy problems despite high accuracy, low entropy on problems with medium difficulty, and high entropy on hard problems reflecting uncertainty. Specifically, we notice 22--25\% entropy reduction from easy to medium difficulty regions, suggesting an {overthinking} phenomenon on easy instances. Building on these insights, we introduce \textbf{DiffAdapt}, a lightweight framework that selects Easy/Normal/Hard inference strategies per question based on their difficulty and reasoning trace entropy. Each inference strategy consists of a fixed prompt, temperature and maximum token length. In contrast to existing efficiency optimization methods, our approach does not fine-tune base LLM but a small probe that classifies LLM's final hidden state, allowing inexpensive adaptation. We comprehensively evaluate our method on five models and eight benchmarks. Our method achieves comparable or improved accuracy while reducing token usage by up to 22.4\%, establishing a practical path toward compute-efficient reasoning.

Figures (10)

• Figure 1: Visualization of model accuracy (blue bar), generation entropy (red line) per difficulty of question (x-axis), across three models (model name at the bottom of the graph). We observe a consistent U-shaped entropy curve along the difficulty levels on multiple models.
• Figure 2: Our three inference strategy configurations. $|$Max$|$ refers to the predefined maximum token budget for generation. Full prompts and detailed configurations can be found in the Appendix \ref{['appendix:complete_reasoning_strategy']}.
• Figure 4: Overview of the DiffAdapt framework. Top: (i) Data Generation: We sample multiple responses from the proxy model, and compute statistics to heurstically assign inference strategy. This process yields a training dataset for the probe; (ii) Probe Training: We train lightweight probe which takes model's hidden states after processing the query and predict its inference strategy label; and (iii) Inference with Probe: where the trained probe dynamically selects appropriate inference strategies (Easy/Normal/Hard).
• Figure 5: Performance across reasoning LLMs model architectures and domains. The x-axis represents different maximum token limit constraints as a percentage of the full token budget, demonstrating how different strategies perform under varying computational budgets. (a) In-domain: DiffAdapt consistently outperforms fixed strategies. (b) Out-of-domain: Effectiveness maintained under domain shift.
• Figure 6: DiffAdapt orthogonality with Length Control RL methods. Performance analysis across three LC-RL trained models on both ID and OOD datasets.