Adaptive Reasoning Executor: A Collaborative Agent System for Efficient Reasoning

TL;DR

The paper tackles the high computational expense of deep reasoning in large language models (LLMs) by introducing a collaborative agent system that leverages both small and large LLMs. The small LLM first generates an initial answer $R_t$ for a question $Q_t$, with a prompt $P_t$ formed by combining $Q_t$ and $R_t$, which the large LLM then verifies; if the answer is not validated, the large LLM performs in-depth reasoning to produce the final result. The authors propose two judgment strategies—Immediate and Step-by-step—to decide whether to accept the small LLM’s answer or require further reasoning, and they validate the approach on GSM8K and MMLU datasets using Qwen and Llama as small LLMs and Gemini as the large LLM. Experimental results show that the method can reduce large-LLM token usage by approximately 50% on simple tasks with only minor accuracy loss, while maintaining robustness on complex tasks; for more challenging problems, incorporating verified steps can further reduce cost by around 20% without sacrificing accuracy. Overall, this work provides a practical, cost-aware framework for adaptive reasoning in multi-LLM settings, with clear trade-offs between evaluation overhead and accuracy depending on task difficulty.

Abstract

Recent advances in Large Language Models (LLMs) demonstrate that chain-of-thought prompting and deep reasoning substantially enhance performance on complex tasks, and multi-agent systems can further improve accuracy by enabling model debates. However, applying deep reasoning to all problems is computationally expensive. To mitigate these costs, we propose a complementary agent system integrating small and large LLMs. The small LLM first generates an initial answer, which is then verified by the large LLM. If correct, the answer is adopted directly; otherwise, the large LLM performs in-depth reasoning. Experimental results show that, for simple problems, our approach reduces the computational cost of the large LLM by more than 50% with negligible accuracy loss, while consistently maintaining robust performance on complex tasks.

Figures (3)

• Figure 1: In our pipeline, the small LLM first generates an initial answer, which is then verified by the large LLM. If the initial answer is judged to be correct, it is adopted directly. Otherwise, the large LLM conducts in-depth reasoning to produce the final result.
• Figure 2: Two judgment strategies in our system. (1) Immediate judgment: the large LLM gives a direct judgment on whether the answer is correct. (2) Step-by-step judgment: Each step of the answer is labeled as [CORRECT] or [INCORRECT], and the process stops once an [INCORRECT] label appears.
• Figure 3: Incorporating verified correct steps into the prompt involves selecting only sentences labeled with [CORRECT], appending them after the question, and providing them to the large model for in-depth reasoning.