Literature Review Of Multi-Agent Debate For Problem-Solving
Arne Tillmann
TL;DR
This literature review surveys multi-agent large language models (MA-LLMs) for problem-solving, focusing on agent profiles, communication topologies, decision-making methods, and scaling. It synthesizes traditional multi-agent taxonomies with MA-LLM-specific findings, highlighting how topology, prompts, and interaction patterns shape performance, while noting substantial costs and biases. The paper identifies gaps such as limited direct cross-system comparisons, insufficient statistical analyses, and under-explored data-driven or reinforcement learning approaches, proposing a roadmap including dynamic networks and robust evaluation. It contributes a structured taxonomy and a consolidated view of how scaling, debate length, and agent diversity interact to influence robustness and efficiency. The practical impact lies in guiding researchers toward cost-aware, scalable, and fair MA-LLM architectures for complex tasks.
Abstract
Multi-agent large language models (MA-LLMs) are a rapidly growing research area that leverages multiple interacting language agents to tackle complex tasks, outperforming single-agent large language models. This literature review synthesizes the latest research on agent profiles, communication structures, and decision-making processes, drawing insights from both traditional multi-agent systems and state-of-the-art MA-LLM studies. In doing so, it aims to address the lack of direct comparisons in the field, illustrating how factors like scalability, communication structure, and decision-making processes influence MA-LLM performance. By examining frequent practices and outlining current challenges, the review reveals that multi-agent approaches can yield superior results but also face elevated computational costs and under-explored challenges unique to MA-LLM. Overall, these findings provide researchers and practitioners with a roadmap for developing robust and efficient multi-agent AI solutions.