Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MasRouter: Learning to Route LLMs for Multi-Agent Systems

Yanwei Yue, Guibin Zhang, Boyang Liu, Guancheng Wan, Kun Wang, Dawei Cheng, Yiyan Qi

TL;DR

This work formalizes Multi-Agent System Routing (MASR) and presents MasRouter, a cascaded controller framework that jointly selects collaboration modes, allocates agent roles, and routes LLM backbones for each query. MasRouter uses a variational latent model for collaboration mode inference, a cascaded inference process for roles, and a multinomial LLM routing mechanism optimized via policy gradient to balance performance and token cost. Empirical results across five benchmarks show MasRouter achieving state-of-the-art or near state-of-the-art accuracy with substantial cost savings, while demonstrating inductive generalization to unseen models and seamless plug-in integration with existing MAS. The approach advances the scalability and practicality of MAS by enabling adaptive, task-aware composition of heterogeneous LLMs with economical resource usage.

Abstract

Multi-agent systems (MAS) powered by Large Language Models (LLMs) have been demonstrated to push the boundaries of LLM capabilities, yet they often incur significant costs and face challenges in dynamic LLM selection. Current LLM routing methods effectively reduce overhead in single-agent scenarios by customizing LLM selection for each query, but they overlook the critical decisions regarding collaboration modes and agent roles in MAS. In response to this challenge, we first introduce the problem of Multi-Agent System Routing (MASR), which integrates all components of MAS into a unified routing framework. Toward this goal, we propose MasRouter, the first high-performing, cost-effective, and inductive MASR solution. MasRouter employs collaboration mode determination, role allocation, and LLM routing through a cascaded controller network, progressively constructing a MAS that balances effectiveness and efficiency. Extensive experiments demonstrate that MasRouter is (1) high-performing, achieving a $1.8\%\sim8.2\%$ improvement over the state-of-the-art method on MBPP; (2) economical, reducing overhead by up to $52.07\%$ compared to SOTA methods on HumanEval; and (3) plug-and-play, seamlessly integrating with mainstream MAS frameworks, reducing overhead by $17.21\%\sim28.17\%$ via customized routing. The code is available at https://github.com/yanweiyue/masrouter.

MasRouter: Learning to Route LLMs for Multi-Agent Systems

TL;DR

This work formalizes Multi-Agent System Routing (MASR) and presents MasRouter, a cascaded controller framework that jointly selects collaboration modes, allocates agent roles, and routes LLM backbones for each query. MasRouter uses a variational latent model for collaboration mode inference, a cascaded inference process for roles, and a multinomial LLM routing mechanism optimized via policy gradient to balance performance and token cost. Empirical results across five benchmarks show MasRouter achieving state-of-the-art or near state-of-the-art accuracy with substantial cost savings, while demonstrating inductive generalization to unseen models and seamless plug-in integration with existing MAS. The approach advances the scalability and practicality of MAS by enabling adaptive, task-aware composition of heterogeneous LLMs with economical resource usage.

Abstract

Multi-agent systems (MAS) powered by Large Language Models (LLMs) have been demonstrated to push the boundaries of LLM capabilities, yet they often incur significant costs and face challenges in dynamic LLM selection. Current LLM routing methods effectively reduce overhead in single-agent scenarios by customizing LLM selection for each query, but they overlook the critical decisions regarding collaboration modes and agent roles in MAS. In response to this challenge, we first introduce the problem of Multi-Agent System Routing (MASR), which integrates all components of MAS into a unified routing framework. Toward this goal, we propose MasRouter, the first high-performing, cost-effective, and inductive MASR solution. MasRouter employs collaboration mode determination, role allocation, and LLM routing through a cascaded controller network, progressively constructing a MAS that balances effectiveness and efficiency. Extensive experiments demonstrate that MasRouter is (1) high-performing, achieving a improvement over the state-of-the-art method on MBPP; (2) economical, reducing overhead by up to compared to SOTA methods on HumanEval; and (3) plug-and-play, seamlessly integrating with mainstream MAS frameworks, reducing overhead by via customized routing. The code is available at https://github.com/yanweiyue/masrouter.

Paper Structure

This paper contains 38 sections, 13 equations, 6 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multi-Agent System.
  4. Single LLM Routing
  5. Formalization
  6. Notation Establishment
  7. Search Space.
  8. Definition of MASR
  9. Optimization Objective.
  10. MasRouter
  11. Collaboration Mode Determination
  12. Agent Role Allocation
  13. Agent LLM Routing
  14. Optimization
  15. Experiments
  16. ...and 23 more sections

Figures (6)

  • Figure 1: Paradigm comparison between single-agent routing and multi-agent routing.
  • Figure 2: The overall framework of our proposed MasRouter.
  • Figure 3: The comparison of the performance and inference cost on the MBPP dataset. Different shapes of the scatter points represent various types of baselines, while the different colors of the points indicate the use of different LLM backbones.
  • Figure 4: The selected LLM distribution of MasRouter on MMLU and MATH benchmark.
  • Figure 5: Sensitivity analysis of MasRouter on HumanEval. The unit of cost per query (right) and performance (left) is $10^{-3}\cdot\$$ and $pass@1~(\%)$, respectively.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Definition 1: Multi-Agent System
  • Definition 2: MASR