AnyMAC: Cascading Flexible Multi-Agent Collaboration via Next-Agent Prediction

TL;DR

AnyMAC reframes multi-agent collaboration from fixed graph topologies to a sequential routing paradigm, enabling Next-Agent Prediction and Next-Context Selection to flexibly allocate roles and retrieve global history. Built on a transformer-based encoder with task-adaptive NAP/NCS tokens, it constructs task-specific communication pipelines that permit agent reuse and global context routing, optimizing for accuracy, efficiency, and robustness via RL training. Empirical results show state-of-the-art or near-state-of-the-art performance across reasoning, math, code, and evaluation benchmarks, with tunable efficiency through the AnyMAC-Eff variant. The approach demonstrates strong resilience to adversarial inputs and provides a scalable framework for flexible, adaptive, and interpretable multi-agent LLM systems in real-world tasks.

Abstract

Recent progress in large language model (LLM)-based multi-agent collaboration highlights the power of structured communication in enabling collective intelligence. However, existing methods largely rely on static or graph-based inter-agent topologies, lacking the potential adaptability and flexibility in communication. In this work, we propose a new framework that rethinks multi-agent coordination through a sequential structure rather than a graph structure, offering a significantly larger topology space for multi-agent communication. Our method focuses on two key directions: (1) Next-Agent Prediction, which selects the most suitable agent role at each step, and (2) Next-Context Selection (NCS), which enables each agent to selectively access relevant information from any previous step. Together, these components construct task-adaptive communication pipelines that support both role flexibility and global information flow. Extensive evaluations across multiple benchmarks demonstrate that our approach achieves superior performance while substantially reducing communication overhead.

Figures (8)

• Figure 1: Comparison of LLM-based multi-agent communication topology design.
• Figure 2: The overview of our proposed framework AnyMAC. Left-hand side: At each time step, we perform two stages of operations: (1) Next-Agent Prediction (NAP), which aims to select the most suitable agent role from a set of candidate roles. (2) Next-Context Selection (NCS), which aims to retrieve useful context from the outputs of previously activated agents. The retrieved context will act as the input to the selected agent. Right-hand side: Given the embeddings of a series of activated agents, we perform contextual encoding using a transformer-based model to encode them with additional NAP and NCS tokens. The output embeddings of NAP and NCS tokens will be used to select the next agent and retrieve context from the next agent, respectively.
• Figure 3: Robustness analysis of different methods on the MMLU dataset. We compare accuracy before and after the attack. Learning-based methods exhibit strong resilience to malicious agents.
• Figure 4: The performance and token consumption of various multi-agent communication topologies on GSM8K.
• Figure 6: Qualitative comparison between AnyMAC (top) and AnyMAC-Eff. (bottom) on the same question. They use different computation budgets, and the increased computation in AnyMAC leads to the correct answer.