AutoMaAS: Self-Evolving Multi-Agent Architecture Search for Large Language Models

TL;DR

AutoMaAS tackles the problem of rigid, static multi-agent architectures for LLMs by introducing a self-evolving NAS-inspired framework that learns a conditional distribution $P(G|q,\theta)$ over architectures. It integrates dynamic operator lifecycle management, multi-objective cost optimization, online feedback, and interpretability to adapt architectures to task complexity and deployment conditions. Empirically, it achieves $1.0\%-7.1\%$ accuracy gains and $3\%-5\%$ cost reductions across six benchmarks, while showing strong transferability across datasets and LLM backbones. This approach advances automated design of adaptive, cost-aware multi-agent systems in the era of large language models and opens avenues for multimodal and federated deployments.

Abstract

Multi-agent systems powered by large language models have demonstrated remarkable capabilities across diverse domains, yet existing automated design approaches seek monolithic solutions that fail to adapt resource allocation based on query complexity and domain requirements. This paper introduces AutoMaAS, a self-evolving multi-agent architecture search framework that leverages neural architecture search principles to automatically discover optimal agent configurations through dynamic operator lifecycle management and automated machine learning techniques. Our approach incorporates four key innovations: (1) automatic operator generation, fusion, and elimination based on performance-cost analysis, (2) dynamic cost-aware optimization with real-time parameter adjustment, (3) online feedback integration for continuous architecture refinement, and (4) enhanced interpretability through decision tracing mechanisms. Extensive experiments across six benchmarks demonstrate that AutoMaAS achieves 1.0-7.1\% performance improvement while reducing inference costs by 3-5\% compared to state-of-the-art methods. The framework shows superior transferability across datasets and LLM backbones, establishing a new paradigm for automated multi-agent system design in the era of large language models.

Figures (5)

• Figure 1: Overall Architecture of AutoMaAS Framework. The system dynamically samples query-dependent multi-agent architectures from an evolving supernet, with continuous operator lifecycle management and automated machine learning optimization.
• Figure 2: Dynamic Operator Lifecycle Management Process. Operators are continuously monitored, evaluated, and evolved based on performance metrics and collaboration patterns.
• Figure 3: Multi-Objective Dynamic Cost Optimization. The cost tensor aggregates multiple dimensions with adaptive weights that respond to real-time system conditions and query characteristics.
• Figure 4: Operator Evolution During Training. The system automatically generates fused operators while maintaining a stable pool of active operators through intelligent elimination.
• Figure 5: Cost-Performance Trade-offs Across Different Methods. AutoMaAS achieves the best balance between accuracy and computational efficiency, positioned in the optimal lower-right region (high accuracy, low cost).