AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgeting

TL;DR

AdaCtrl tackles the inefficiency of deep reasoning in large language models by introducing adaptive and controllable reasoning via difficulty-aware budgeting. It couples a cold-start fine-tuning stage with a difficulty-aware reinforcement learning phase to calibrate problem difficulty and dynamically allocate reasoning budgets, while enabling user control through explicit [Easy]/[Hard] tags. The approach achieves competitive accuracy while significantly reducing response length across four math benchmarks, and demonstrates robust controllability and better difficulty estimation. This framework offers a practical path to balancing reasoning quality and computational efficiency in real-world, user-driven tasks.

Abstract

Modern large reasoning models demonstrate impressive problem-solving capabilities by employing sophisticated reasoning strategies. However, they often struggle to balance efficiency and effectiveness, frequently generating unnecessarily lengthy reasoning chains for simple problems. In this work, we propose AdaCtrl, a novel framework to support both difficulty-aware adaptive reasoning budget allocation and explicit user control over reasoning depth. AdaCtrl dynamically adjusts its reasoning length based on self-assessed problem difficulty, while also allowing users to manually control the budget to prioritize either efficiency or effectiveness. This is achieved through a two-stage training pipeline: an initial cold-start fine-tuning phase to instill the ability to self-aware difficulty and adjust reasoning budget, followed by a difficulty-aware reinforcement learning (RL) stage that refines the model's adaptive reasoning strategies and calibrates its difficulty assessments based on its evolving capabilities during online training. To enable intuitive user interaction, we design explicit length-triggered tags that function as a natural interface for budget control. Empirical results show that AdaCtrl adapts reasoning length based on estimated difficulty, compared to the standard training baseline that also incorporates fine-tuning and RL, it yields performance improvements and simultaneously reduces response length by 10.06% and 12.14% on the more challenging AIME2024 and AIME2025 datasets, which require elaborate reasoning, and by 62.05% and 91.04% on the MATH500 and GSM8K datasets, where more concise responses are sufficient. Furthermore, AdaCtrl enables precise user control over the reasoning budget, allowing for tailored responses to meet specific needs.

Figures (6)

• Figure 1: Given the same problem, AdaCtrl supports three reasoning modes: the easy mode offers concise answers with less tokens; the hard mode delivers extensive responses with more tokens; and the adaptive mode dynamically allocates reasoning budgets according to the problem complexity.
• Figure 2: AdaCtrl comprises a two-stage training pipeline: the cold-start finetuning first utilizes both short and long reasoning trajectories to establish basic budget awareness; then a difficulty-aware reinforcement learning framework is utilized to calibrate problem difficulty estimation and develop adaptive reasoning strategies.
• Figure 3: The proportion dynamics of difficulty-aware tags across different datasets during reinforcement learning.
• Figure 4: (a) The length of response in different difficulty levels of problems in MATH500, where higher levels indicate more challenging problems; (b) Controllability comparison of AdaCtrl and prompting-based approach.
• Figure 5: The length distribution of response generated by AdaCtrl-7B.