Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-agent Architecture Search via Agentic Supernet

Guibin Zhang, Luyang Niu, Junfeng Fang, Kun Wang, Lei Bai, Xiang Wang

TL;DR

MaAS reframes automated multi-agent system design as learning a probabilistic distribution over architectures via an agentic supernet. It uses a query-conditioned controller to sample tailored, resource-aware multi-agent configurations, optimizing a trade-off between performance and cost through Monte Carlo and textual gradient updates. Empirical results across math, code, and tool-use benchmarks show improved accuracy and significantly lower training/inference costs, with strong transferability across datasets and backbones. The approach enables dynamic, self-evolving collective intelligence and has potential to democratize access to advanced automated reasoning systems.

Abstract

Large Language Model (LLM)-empowered multi-agent systems extend the cognitive boundaries of individual agents through disciplined collaboration and interaction, while constructing these systems often requires labor-intensive manual designs. Despite the availability of methods to automate the design of agentic workflows, they typically seek to identify a static, complex, one-size-fits-all system, which, however, fails to dynamically allocate inference resources based on the difficulty and domain of each query. To address this challenge, we shift away from the pursuit of a monolithic agentic system, instead optimizing the \textbf{agentic supernet}, a probabilistic and continuous distribution of agentic architectures. We introduce MaAS, an automated framework that samples query-dependent agentic systems from the supernet, delivering high-quality solutions and tailored resource allocation (\textit{e.g.}, LLM calls, tool calls, token cost). Comprehensive evaluation across six benchmarks demonstrates that MaAS \textbf{(I)} requires only $6\sim45\%$ of the inference costs of existing handcrafted or automated multi-agent systems, \textbf{(II)} surpasses them by $0.54\%\sim11.82\%$, and \textbf{(III)} enjoys superior cross-dataset and cross-LLM-backbone transferability.

Multi-agent Architecture Search via Agentic Supernet

TL;DR

MaAS reframes automated multi-agent system design as learning a probabilistic distribution over architectures via an agentic supernet. It uses a query-conditioned controller to sample tailored, resource-aware multi-agent configurations, optimizing a trade-off between performance and cost through Monte Carlo and textual gradient updates. Empirical results across math, code, and tool-use benchmarks show improved accuracy and significantly lower training/inference costs, with strong transferability across datasets and backbones. The approach enables dynamic, self-evolving collective intelligence and has potential to democratize access to advanced automated reasoning systems.

Abstract

Large Language Model (LLM)-empowered multi-agent systems extend the cognitive boundaries of individual agents through disciplined collaboration and interaction, while constructing these systems often requires labor-intensive manual designs. Despite the availability of methods to automate the design of agentic workflows, they typically seek to identify a static, complex, one-size-fits-all system, which, however, fails to dynamically allocate inference resources based on the difficulty and domain of each query. To address this challenge, we shift away from the pursuit of a monolithic agentic system, instead optimizing the \textbf{agentic supernet}, a probabilistic and continuous distribution of agentic architectures. We introduce MaAS, an automated framework that samples query-dependent agentic systems from the supernet, delivering high-quality solutions and tailored resource allocation (\textit{e.g.}, LLM calls, tool calls, token cost). Comprehensive evaluation across six benchmarks demonstrates that MaAS \textbf{(I)} requires only of the inference costs of existing handcrafted or automated multi-agent systems, \textbf{(II)} surpasses them by , and \textbf{(III)} enjoys superior cross-dataset and cross-LLM-backbone transferability.

Paper Structure

This paper contains 36 sections, 12 equations, 10 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM-Agents and Agentic Systems.
  4. Automating Agentic Systems.
  5. AutoML.
  6. Methodology
  7. Preliminary
  8. Search Space.
  9. Problem Formulation.
  10. Agentic Architecture Sampling
  11. Cost-constrained Supernet Optimization
  12. Experiments
  13. Experiment Setup
  14. Tasks and Benchmarks.
  15. Baselines.
  16. ...and 21 more sections

Figures (10)

  • Figure 1: (Left) The building blocks of MaAS; (Right) When confronting different queries, the agentic supernet adaptively samples tailored multi-agent architecture in a query-dependent manner.
  • Figure 2: The overall framework of our proposed MaAS.
  • Figure 3: The demonstration of textual gradient.
  • Figure 4: The cost analysis of MaAS on MATH benchmark.
  • Figure 5: The visualization of MaAS's operator sampling process.
  • ...and 5 more figures

Theorems & Definitions (2)

  • Definition 3.1: Agentic Operator
  • Definition 3.2: Agentic Supernet