DynaDebate: Breaking Homogeneity in Multi-Agent Debate with Dynamic Path Generation

TL;DR

DynaDebate tackles initialization homogeneity in multi-agent debate by introducing dynamic path generation to diversify agents' reasoning paths, coupled with a process-centric debate that audits step-by-step logic. A Trigger-Based Verification Agent uses external tools to resolve deadlocks and provide objective references, reducing blind conformity. Across six benchmarks and with both GPT-4o-mini and Qwen3-8B, DynaDebate delivers superior or highly competitive results, especially on challenging mathematical reasoning tasks, and even smaller models can surpass larger baselines when equipped with the framework. The work highlights the value of explicit diversity in reasoning paths together with rigorous verification for robust collaborative problem solving in MAD systems.

Abstract

Recent years have witnessed the rapid development of Large Language Model-based Multi-Agent Systems (MAS), which excel at collaborative decision-making and complex problem-solving. Recently, researchers have further investigated Multi-Agent Debate (MAD) frameworks, which enhance the reasoning and collaboration capabilities of MAS through information exchange and debate among multiple agents. However, existing approaches often rely on unguided initialization, causing agents to adopt identical reasoning paths that lead to the same errors. As a result, effective debate among agents is hindered, and the final outcome frequently degenerates into simple majority voting. To solve the above problem, in this paper, we introduce Dynamic Multi-Agent Debate (DynaDebate), which enhances the effectiveness of multi-agent debate through three key mechanisms: (1) Dynamic Path Generation and Allocation, which employs a dedicated Path Generation Agent to generate diverse and logical solution paths with adaptive redundancy; (2) Process-Centric Debate, which shifts the focus from surface-level outcome voting to rigorous step-by-step logic critique to ensure process correctness; (3) A Trigger-Based Verification Agent, which is activated upon disagreement and uses external tools to objectively resolve deadlocks. Extensive experiments demonstrate that DynaDebate achieves superior performance across various benchmarks, surpassing existing state-of-the-art MAD methods.

Figures (6)

• Figure 1: Comparison of reasoning paths in the multi-agent debate phase. (Top) Existing methods: Agents participating in the debate follow identical reasoning paths. (Bottom) In the DynaDebate framework, agents are assigned distinct reasoning paths, ensuring diversity in reasoning paths.
• Figure 2: The overall framework of DynaDebate. The process operates in three phases: (1) Dynamic Path Generation: Initializes agents with diverse, logically sound paths to break homogeneity. (2) Process-Centric Debate: Agents perform a First-Principles Audit on peer reasoning steps. (3) Trigger-Based Verification: A Verification Agent integrates external tools to provide a reference for the debate and help resolve deadlocks.
• Figure 3: Diversity analysis of multi-agent reasoning. (a) Intra-diversity measures semantic variation among agents. (b) Structural non-overlap measures diversity at the reasoning-structure level. Our method consistently improves both metrics across datasets, indicating structured and effective diversification.
• Figure 4: Performance comparison under different numbers of agents. Results are reported for multiple benchmarks, with each subplot showing the performance trends as the number of agents increases.
• Figure 5: Effect of the number of debate rounds $N$ on performance across benchmarks.