Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic Role Assignment for Multi-Agent Debate

Miao Zhang, Junsik Kim, Siyuan Xiang, Jian Gao, Cheng Cao

TL;DR

This work proposes dynamic role assignment, a framework that runs a Meta-Debate to select suitable agents before the actual debate to establish a new paradigm for multi-agent system design, shifting from static agent deployment to dynamic and capability-aware selection.

Abstract

Multi-agent large language model (LLM) and vision-language model (VLM) debate systems employ specialized roles for complex problem-solving, yet model specializations are not leveraged to decide which model should fill which role. We propose dynamic role assignment, a framework that runs a Meta-Debate to select suitable agents before the actual debate. The meta-debate has two stages: (1) proposal, where candidates provide role-tailored arguments, and (2) peer review, where proposals are scored with data and role-specific criteria to choose the best agent for each position. We evaluate our method on LLM problem solving benchmarks. Applied on top of existing debate systems, our approach consistently outperforms uniform assignments (filling all roles with the same model) by up to 74.8% and random assignments (assigning models to roles without considering their suitability) by up to 29.7%, depending on the task and the specific assignment. This work establishes a new paradigm for multi-agent system design, shifting from static agent deployment to dynamic and capability-aware selection.

Dynamic Role Assignment for Multi-Agent Debate

TL;DR

This work proposes dynamic role assignment, a framework that runs a Meta-Debate to select suitable agents before the actual debate to establish a new paradigm for multi-agent system design, shifting from static agent deployment to dynamic and capability-aware selection.

Abstract

Multi-agent large language model (LLM) and vision-language model (VLM) debate systems employ specialized roles for complex problem-solving, yet model specializations are not leveraged to decide which model should fill which role. We propose dynamic role assignment, a framework that runs a Meta-Debate to select suitable agents before the actual debate. The meta-debate has two stages: (1) proposal, where candidates provide role-tailored arguments, and (2) peer review, where proposals are scored with data and role-specific criteria to choose the best agent for each position. We evaluate our method on LLM problem solving benchmarks. Applied on top of existing debate systems, our approach consistently outperforms uniform assignments (filling all roles with the same model) by up to 74.8% and random assignments (assigning models to roles without considering their suitability) by up to 29.7%, depending on the task and the specific assignment. This work establishes a new paradigm for multi-agent system design, shifting from static agent deployment to dynamic and capability-aware selection.
Paper Structure (20 sections, 5 equations, 2 figures, 2 tables)

This paper contains 20 sections, 5 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM based multi-agent debate.
  4. Automated design of multi-agent systems.
  5. Open challenges.
  6. Methods
  7. Meta-Debate process
  8. Generation of Agent Evaluation Criteria
  9. Experiments
  10. Debate baselines
  11. Benchmarks
  12. Results
  13. Model debate competency varies
  14. Role assignment effects
  15. Dynamic versus static allocation
  16. ...and 5 more sections

Figures (2)

  • Figure 1: Current LLM based multi-agent debate methods either do not explicitly define agent role functions (a), or use identical model in all roles without assignment (b). In contrast, our method dynamically assigns different models to roles based on tasks (c), matching their strengths to role requirements on each question, leading more effective and optimized collaboration.
  • Figure 2: The Meta-Debate framework for role selection consists of two steps. (1) Each agent is first prompted to generate a role-specific response, referred to as a "proposal". (2) Then, for each role, all agents evaluate and score the proposals according to predefined criteria. The average of these scores is taken as the final score of each agent for that role for solving the specific question. The agent with the highest score is ultimately assigned to the role, and a same agent can be assigned to multiple roles.