Paper
Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberation
Authors
Zhiwei Zhang, Xiaomin Li, Yudi Lin, Hui Liu, Ramraj Chandradevan, Linlin Wu, Minhua Lin, Fali Wang, Xianfeng Tang, Qi He, Suhang Wang
Abstract
Large Language Models (LLMs) trained with reinforcement learning and
verifiable rewards have achieved strong results on complex reasoning tasks.
Recent work extends this paradigm to a multi-agent setting, where a
meta-thinking agent proposes plans and monitors progress while a reasoning
agent executes subtasks through sequential conversational turns. Despite
promising performance, we identify a critical limitation: lazy agent behavior,
in which one agent dominates while the other contributes little, undermining
collaboration and collapsing the setup to an ineffective single agent. In this
paper, we first provide a theoretical analysis showing why lazy behavior
naturally arises in multi-agent reasoning. We then introduce a stable and
efficient method for measuring causal influence, helping mitigate this issue.
Finally, as collaboration intensifies, the reasoning agent risks getting lost
in multi-turn interactions and trapped by previous noisy responses. To counter
this, we propose a verifiable reward mechanism that encourages deliberation by
allowing the reasoning agent to discard noisy outputs, consolidate
instructions, and restart its reasoning process when necessary. Extensive
experiments demonstrate that our framework alleviates lazy agent behavior and
unlocks the full potential of multi-agent framework for complex reasoning
tasks.