Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OFA-MAS: One-for-All Multi-Agent System Topology Design based on Mixture-of-Experts Graph Generative Models

Shiyuan Li, Yixin Liu, Yu Zheng, Mei Li, Quoc Viet Hung Nguyen, Shirui Pan

TL;DR

This work tackles the challenge of designing adaptive collaboration topologies for multi-agent systems across diverse tasks. It introduces OFA-MAS, a one-for-all topology designer that combines a Task-Aware Graph State Encoder and a Mixture-of-Experts generation module, trained via a three-stage curriculum including LLM-driven data synthesis. The approach enables autoregressive generation of task-specific graphs from a universal role pool, achieving state-of-the-art results on six benchmarks and strong zero-shot generalization to unseen domains. By leveraging cross-domain shared patterns and expert specialization, OFA-MAS offers a scalable alternative to domain-specific topology design with practical implications for robust, cross-domain MAS deployments.

Abstract

Multi-Agent Systems (MAS) offer a powerful paradigm for solving complex problems, yet their performance is critically dependent on the design of their underlying collaboration topology. As MAS become increasingly deployed in web services (e.g., search engines), designing adaptive topologies for diverse cross-domain user queries becomes essential. Current graph learning-based design methodologies often adhere to a "one-for-one" paradigm, where a specialized model is trained for each specific task domain. This approach suffers from poor generalization to unseen domains and fails to leverage shared structural knowledge across different tasks. To address this, we propose OFA-TAD, a one-for-all framework that generates adaptive collaboration graphs for any task described in natural language through a single universal model. Our approach integrates a Task-Aware Graph State Encoder (TAGSE) that filters task-relevant node information via sparse gating, and a Mixture-of-Experts (MoE) architecture that dynamically selects specialized sub-networks to drive node and edge prediction. We employ a three-stage training strategy: unconditional pre-training on canonical topologies for structural priors, large-scale conditional pre-training on LLM-generated datasets for task-topology mappings, and supervised fine-tuning on empirically validated graphs. Experiments across six diverse benchmarks show that OFA-TAD significantly outperforms specialized one-for-one models, generating highly adaptive MAS topologies. Code: https://github.com/Shiy-Li/OFA-MAS.

OFA-MAS: One-for-All Multi-Agent System Topology Design based on Mixture-of-Experts Graph Generative Models

TL;DR

This work tackles the challenge of designing adaptive collaboration topologies for multi-agent systems across diverse tasks. It introduces OFA-MAS, a one-for-all topology designer that combines a Task-Aware Graph State Encoder and a Mixture-of-Experts generation module, trained via a three-stage curriculum including LLM-driven data synthesis. The approach enables autoregressive generation of task-specific graphs from a universal role pool, achieving state-of-the-art results on six benchmarks and strong zero-shot generalization to unseen domains. By leveraging cross-domain shared patterns and expert specialization, OFA-MAS offers a scalable alternative to domain-specific topology design with practical implications for robust, cross-domain MAS deployments.

Abstract

Multi-Agent Systems (MAS) offer a powerful paradigm for solving complex problems, yet their performance is critically dependent on the design of their underlying collaboration topology. As MAS become increasingly deployed in web services (e.g., search engines), designing adaptive topologies for diverse cross-domain user queries becomes essential. Current graph learning-based design methodologies often adhere to a "one-for-one" paradigm, where a specialized model is trained for each specific task domain. This approach suffers from poor generalization to unseen domains and fails to leverage shared structural knowledge across different tasks. To address this, we propose OFA-TAD, a one-for-all framework that generates adaptive collaboration graphs for any task described in natural language through a single universal model. Our approach integrates a Task-Aware Graph State Encoder (TAGSE) that filters task-relevant node information via sparse gating, and a Mixture-of-Experts (MoE) architecture that dynamically selects specialized sub-networks to drive node and edge prediction. We employ a three-stage training strategy: unconditional pre-training on canonical topologies for structural priors, large-scale conditional pre-training on LLM-generated datasets for task-topology mappings, and supervised fine-tuning on empirically validated graphs. Experiments across six diverse benchmarks show that OFA-TAD significantly outperforms specialized one-for-one models, generating highly adaptive MAS topologies. Code: https://github.com/Shiy-Li/OFA-MAS.
Paper Structure (42 sections, 13 equations, 4 figures, 4 tables)

This paper contains 42 sections, 13 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Formulate MAS as Graphs
  4. From One-for-One to One-for-All Paradigm
  5. Methodology
  6. Autoregressive Graph Generation Backbone
  7. Task-Aware Graph State Encoder
  8. Mixture-of-Experts Generation Module
  9. Expert Gating Network
  10. MoE Prediction Heads
  11. Training Strategy and Data Construction
  12. Stage 1: Unconditional Graph Pre-training
  13. Stage 2: Task-Topology Alignment via LLM-driven Data Synthesis
  14. Stage 3: Supervised Generative Fine-tuning
  15. Training Objective
  16. ...and 27 more sections

Figures (4)

  • Figure 1: Skecth maps of OFA-MAS.
  • Figure 2: The workflow of OFA-MAS.
  • Figure 3: Comprehensive analysis including (a) robustness against adversarial attacks, (b-c) case study visualizations of generated topologies, and (d) MoE expert activation heatmaps across four domains showing clear specialization patterns.
  • Figure 4: The prompt token cost comparison.