Adaptive Dual Reasoner: Large Reasoning Models Can Think Efficiently by Hybrid Reasoning
Yujian Zhang, Keyu Chen, Zhifeng Shen, Ruizhi Qiao, Xing Sun
TL;DR
ADR addresses overthinking in long reasoning models by enabling dynamic switching between fast and slow thinking modes, guided by contextual difficulty. The approach combines a two-stage training pipeline—cold-start supervised fine-tuning on a hybrid reasoning dataset and reinforcement learning with Entropy-guided Hybrid Policy Optimization (EHPO)—to allocate reasoning effort efficiently. A data-construction pipeline labels reasoning units by entropy to selectively compress easy steps and preserve hard steps, while EHPO uses an entropy-aware rollout and a difficulty-aware penalty to balance accuracy and token usage. Experiments on math benchmarks show ADR achieves up to 6.1% accuracy gains with substantial reductions in output length (roughly 50–60%), achieving strong efficiency-accuracy trade-offs across tasks of varying difficulty.
Abstract
Although Long Reasoning Models (LRMs) have achieved superior performance on various reasoning scenarios, they often suffer from increased computational costs and inference latency caused by overthinking. To address these limitations, we propose Adaptive Dual Reasoner, which supports two reasoning modes: fast thinking and slow thinking. ADR dynamically alternates between these modes based on the contextual complexity during reasoning. ADR is trained in two stages: (1) A cold-start stage using supervised fine-tuning (SFT) to equip the model with the ability to integrate both fast and slow reasoning modes, in which we construct a hybrid reasoning dataset through a dedicated pipeline to provide large-scale supervision. (2) A reinforcement learning stage for optimizing reasoning effort, where we introduce Entropy-guided Hybrid Policy Optimization EHPO, an RL training framework employing an entropy-guided dynamic rollout strategy for branching at high-entropy units and a difficulty-aware penalty to balance fast and slow reasoning. Across challenging mathematical reasoning benchmarks, ADR achieves an effective balance between reasoning performance and efficiency among state-of-the-art approaches. Specifically, ADR yields a performance gain of up to 6.1%, while reducing the reasoning output length by 49.5% to 59.3%.