Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SYMPHONY: Synergistic Multi-agent Planning with Heterogeneous Language Model Assembly

Wei Zhu, Zhiwen Tang, Kun Yue

TL;DR

SYMPHONY tackles the limited exploration of single-agent LLM planning by introducing a heterogeneous pool of language-model agents that collaborate within Monte Carlo Tree Search. It combines adaptive UCB-based scheduling, pool-wise memory sharing, and entropy-modulated node evaluation to generate diverse, informative search trajectories. Across HotpotQA, WebShop, and MBPP, SYMPHONY with open-source LLMs demonstrates strong performance and efficiency, with further gains when deployed on cloud-based models; the approach remains cost-effective and adaptable to various hardware. The framework advances practical multi-agent planning with LLMs, offering scalable improvements for complex reasoning, planning, and code-generation tasks while highlighting room for future enhancements in robustness, automation, and fairness.

Abstract

Recent advancements have increasingly focused on leveraging large language models (LLMs) to construct autonomous agents for complex problem-solving tasks. However, existing approaches predominantly employ a single-agent framework to generate search branches and estimate rewards during Monte Carlo Tree Search (MCTS) planning. This single-agent paradigm inherently limits exploration capabilities, often resulting in insufficient diversity among generated branches and suboptimal planning performance. To overcome these limitations, we propose Synergistic Multi-agent Planning with Heterogeneous langauge model assembly (SYMPHONY), a novel multi-agent planning framework that integrates a pool of heterogeneous language model-based agents. By leveraging diverse reasoning patterns across agents, SYMPHONY enhances rollout diversity and facilitates more effective exploration. Empirical results across multiple benchmark tasks show that SYMPHONY achieves strong performance even when instantiated with open-source LLMs deployable on consumer-grade hardware. When enhanced with cloud-based LLMs accessible via API, SYMPHONY demonstrates further improvements, outperforming existing state-of-the-art baselines and underscoring the effectiveness of heterogeneous multi-agent coordination in planning tasks.

SYMPHONY: Synergistic Multi-agent Planning with Heterogeneous Language Model Assembly

TL;DR

SYMPHONY tackles the limited exploration of single-agent LLM planning by introducing a heterogeneous pool of language-model agents that collaborate within Monte Carlo Tree Search. It combines adaptive UCB-based scheduling, pool-wise memory sharing, and entropy-modulated node evaluation to generate diverse, informative search trajectories. Across HotpotQA, WebShop, and MBPP, SYMPHONY with open-source LLMs demonstrates strong performance and efficiency, with further gains when deployed on cloud-based models; the approach remains cost-effective and adaptable to various hardware. The framework advances practical multi-agent planning with LLMs, offering scalable improvements for complex reasoning, planning, and code-generation tasks while highlighting room for future enhancements in robustness, automation, and fairness.

Abstract

Recent advancements have increasingly focused on leveraging large language models (LLMs) to construct autonomous agents for complex problem-solving tasks. However, existing approaches predominantly employ a single-agent framework to generate search branches and estimate rewards during Monte Carlo Tree Search (MCTS) planning. This single-agent paradigm inherently limits exploration capabilities, often resulting in insufficient diversity among generated branches and suboptimal planning performance. To overcome these limitations, we propose Synergistic Multi-agent Planning with Heterogeneous langauge model assembly (SYMPHONY), a novel multi-agent planning framework that integrates a pool of heterogeneous language model-based agents. By leveraging diverse reasoning patterns across agents, SYMPHONY enhances rollout diversity and facilitates more effective exploration. Empirical results across multiple benchmark tasks show that SYMPHONY achieves strong performance even when instantiated with open-source LLMs deployable on consumer-grade hardware. When enhanced with cloud-based LLMs accessible via API, SYMPHONY demonstrates further improvements, outperforming existing state-of-the-art baselines and underscoring the effectiveness of heterogeneous multi-agent coordination in planning tasks.
Paper Structure (40 sections, 1 theorem, 12 equations, 6 figures, 9 tables, 1 algorithm)

This paper contains 40 sections, 1 theorem, 12 equations, 6 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM-based Planning and Reasoning
  4. Multi-Agent Collaboration with Language Models
  5. SYMPHONY
  6. Background and Overview
  7. Heterogeneous Agent Pool
  8. Agent Scheduling
  9. Pool-wise Memory Sharing
  10. Entropy-Modulated Node Evaluation
  11. Experiments
  12. Experiment Settings
  13. Reasoning:HotpotQA
  14. Sequential Decision Making:WebShop
  15. Programming:MBPP
  16. ...and 25 more sections

Key Result

Theorem 1

Consider an agent pool that samples multiple agents with non-zero probabilities. If it satisfies (i) correct coverage, meaning that at each step at least one agent outputs the correct action, and (ii) non-triviality, meaning that no single agent is correct on all steps, then the ensemble achieves a

Figures (6)

  • Figure 1: SYMPHONY System Overview.
  • Figure 2: Branch Diversity vs. Task Performance. Bars and left y-axis shows the branch diversity, while lineplot and right y-axis shows the task performance.
  • Figure 3: Comparison of model invocation frequency and final performance on HotpotQA.
  • Figure 4: Comparison of the search tree size on HotpotQA.
  • Figure 4: Comparison of model invocation frequency and final performance on HotpotQA, highlighting the cost-effectiveness of each method.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Theorem 1: Strict Improvement of Agent Pool Sampling
  • proof