Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reasoning While Asking: Transforming Reasoning Large Language Models from Passive Solvers to Proactive Inquirers

Xin Chen, Feng Jiang, Yiqian Zhang, Hardy Chen, Shuo Yan, Wenya Xie, Min Yang, Shujian Huang

TL;DR

This work addresses the blind self-thinking limitation in reasoning LLMs by introducing Proactive Interactive Reasoning (PIR), which interleaves reasoning with user-driven clarification. PIR consists of two phases: Phase I activates interactive capability through uncertainty-aware data augmentation that converts CoT into think–ask–respond trajectories, and Phase II aligns reasoning with user intent via US-GRPO, a reinforcement-learning framework with a dynamic user simulator and a composite reward balancing correctness, efficiency, and helpful clarifications. Empirical results across mathematical reasoning, code generation, and document editing show PIR achieves state-of-the-art performance and substantially reduces unnecessary interaction, while generalizing to factual knowledge, QA, and missing-premise scenarios. The approach demonstrates that proactive clarification, when properly guided by uncertainty and user intent, improves accuracy and interaction efficiency, offering a robust, human-centric paradigm for next-generation reasoning models.

Abstract

Reasoning-oriented Large Language Models (LLMs) have achieved remarkable progress with Chain-of-Thought (CoT) prompting, yet they remain fundamentally limited by a \emph{blind self-thinking} paradigm: performing extensive internal reasoning even when critical information is missing or ambiguous. We propose Proactive Interactive Reasoning (PIR), a new reasoning paradigm that transforms LLMs from passive solvers into proactive inquirers that interleave reasoning with clarification. Unlike existing search- or tool-based frameworks that primarily address knowledge uncertainty by querying external environments, PIR targets premise- and intent-level uncertainty through direct interaction with the user. PIR is implemented via two core components: (1) an uncertainty-aware supervised fine-tuning procedure that equips models with interactive reasoning capability, and (2) a user-simulator-based policy optimization framework driven by a composite reward that aligns model behavior with user intent. Extensive experiments on mathematical reasoning, code generation, and document editing demonstrate that PIR consistently outperforms strong baselines, achieving up to 32.70\% higher accuracy, 22.90\% higher pass rate, and 41.36 BLEU improvement, while reducing nearly half of the reasoning computation and unnecessary interaction turns. Further reliability evaluations on factual knowledge, question answering, and missing-premise scenarios confirm the strong generalization and robustness of PIR. Model and code are publicly available at: \href{https://github.com/SUAT-AIRI/Proactive-Interactive-R1}

Reasoning While Asking: Transforming Reasoning Large Language Models from Passive Solvers to Proactive Inquirers

TL;DR

This work addresses the blind self-thinking limitation in reasoning LLMs by introducing Proactive Interactive Reasoning (PIR), which interleaves reasoning with user-driven clarification. PIR consists of two phases: Phase I activates interactive capability through uncertainty-aware data augmentation that converts CoT into think–ask–respond trajectories, and Phase II aligns reasoning with user intent via US-GRPO, a reinforcement-learning framework with a dynamic user simulator and a composite reward balancing correctness, efficiency, and helpful clarifications. Empirical results across mathematical reasoning, code generation, and document editing show PIR achieves state-of-the-art performance and substantially reduces unnecessary interaction, while generalizing to factual knowledge, QA, and missing-premise scenarios. The approach demonstrates that proactive clarification, when properly guided by uncertainty and user intent, improves accuracy and interaction efficiency, offering a robust, human-centric paradigm for next-generation reasoning models.

Abstract

Reasoning-oriented Large Language Models (LLMs) have achieved remarkable progress with Chain-of-Thought (CoT) prompting, yet they remain fundamentally limited by a \emph{blind self-thinking} paradigm: performing extensive internal reasoning even when critical information is missing or ambiguous. We propose Proactive Interactive Reasoning (PIR), a new reasoning paradigm that transforms LLMs from passive solvers into proactive inquirers that interleave reasoning with clarification. Unlike existing search- or tool-based frameworks that primarily address knowledge uncertainty by querying external environments, PIR targets premise- and intent-level uncertainty through direct interaction with the user. PIR is implemented via two core components: (1) an uncertainty-aware supervised fine-tuning procedure that equips models with interactive reasoning capability, and (2) a user-simulator-based policy optimization framework driven by a composite reward that aligns model behavior with user intent. Extensive experiments on mathematical reasoning, code generation, and document editing demonstrate that PIR consistently outperforms strong baselines, achieving up to 32.70\% higher accuracy, 22.90\% higher pass rate, and 41.36 BLEU improvement, while reducing nearly half of the reasoning computation and unnecessary interaction turns. Further reliability evaluations on factual knowledge, question answering, and missing-premise scenarios confirm the strong generalization and robustness of PIR. Model and code are publicly available at: \href{https://github.com/SUAT-AIRI/Proactive-Interactive-R1}
Paper Structure (52 sections, 9 equations, 8 figures, 10 tables)

This paper contains 52 sections, 9 equations, 8 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reasoning Large Language Model
  4. Interactive Large Language Model
  5. Proactive Interactive Reasoning Framework
  6. Phase I: Interactive Capability Activation
  7. Uncertainty-Aware Interactive Data Augmentation
  8. Supervised Fine-Tuning for Cold-Start
  9. Phase II: User-Intent Alignment
  10. Group Relative Policy Optimization with User Simulator
  11. User Simulator Construction.
  12. Optimization with GRPO.
  13. Composite Reward Modeling
  14. Output Reward.
  15. Reasoning Reward.
  16. ...and 37 more sections

Figures (8)

  • Figure 1: The Proactive Interactive Reasoning (PIR) Paradigm. The schematic contrasts inefficient "blind self-thinking" on ambiguous queries with the PIR approach. PIR utilizes uncertainty detection and a two-phase optimization mechanism to enable proactive clarification with a user simulator, aligning reasoning chains with user intent to achieve accurate problem-solving that is efficient, robust, and minimal compute costs.
  • Figure 2: Overview of the PIR Framework. The framework operates in two phases to transition LLMs from passive solvers to active inquirers.
  • Figure 3: Comparison of PIR models trained with different reward modeling and user simulators in US-GRPO in Math-Chat.
  • Figure 4: Case study demonstrating user intent: Let x and y be real numbers such that $2(x^2 + y^2)$ = $x + y$. Find the maximum value of $x - y$. The dashed rectangle ( ) denotes the internal reasoning process, while the solid rectangle ( ) denotes the interaction workflow. The PIR LLM shows High Preference and Efficiency compared to the blind Self-Thinking Mode.
  • Figure 5: Uncertainty Analysis on SFT Test Dataset. The left means the distribution of PE values for asking trigger sentences across different dataset sizes. The right trends of Mean PE (left axis) and Asking-Response Template Correctness (right axis), illustrating the correlation between the model's focus on high uncertainty and its structural interactive capability.
  • ...and 3 more figures