Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Latency-Aware Orchestration for Parallel Multi-Agent Systems

Xi Shi, Mengxin Zheng, Qian Lou

TL;DR

Latency is a bottleneck in parallel multi-agent systems, where traditional accuracy or cost-focused objectives fail to guarantee fast end-to-end reasoning. LAMaS introduces a latency-aware orchestration framework that learns layer-wise, parallel execution topologies on a probabilistic DAG supernet and uses a critical-path-aware reward to minimize the longest dependency chain. Empirical results across GSM8K, HumanEval, and MATH show a 38–46% reduction in the critical path length with comparable or improved task performance, demonstrating practical gains for latency-sensitive MAS. This work provides a principled approach to balancing parallelism, reasoning depth, and latency, paving the way for production-ready, latency-aware MAS deployments.

Abstract

Multi-agent systems (MAS) enable complex reasoning by coordinating multiple agents, but often incur high inference latency due to multi-step execution and repeated model invocations, severely limiting their scalability and usability in time-sensitive scenarios. Most existing approaches primarily optimize task performance and inference cost, and explicitly or implicitly assume sequential execution, making them less optimal for controlling latency under parallel execution. In this work, we investigate learning-based orchestration of multi-agent systems with explicit latency supervision under parallel execution. We propose Latency-Aware Multi-agent System (LAMaS), a latency-aware multi-agent orchestration framework that enables parallel execution and explicitly optimizes the critical execution path, allowing the controller to construct execution topology graphs with lower latency under parallel execution. Our experiments show that our approach reduces critical path length by 38-46% compared to the state-of-the-art baseline for multi-agent architecture search across multiple benchmarks, while maintaining or even improving task performance. These results highlight the importance of explicitly optimizing latency under parallel execution when designing efficient multi-agent systems. The code is available at https://github.com/xishi404/LAMaS

Learning Latency-Aware Orchestration for Parallel Multi-Agent Systems

TL;DR

Latency is a bottleneck in parallel multi-agent systems, where traditional accuracy or cost-focused objectives fail to guarantee fast end-to-end reasoning. LAMaS introduces a latency-aware orchestration framework that learns layer-wise, parallel execution topologies on a probabilistic DAG supernet and uses a critical-path-aware reward to minimize the longest dependency chain. Empirical results across GSM8K, HumanEval, and MATH show a 38–46% reduction in the critical path length with comparable or improved task performance, demonstrating practical gains for latency-sensitive MAS. This work provides a principled approach to balancing parallelism, reasoning depth, and latency, paving the way for production-ready, latency-aware MAS deployments.

Abstract

Multi-agent systems (MAS) enable complex reasoning by coordinating multiple agents, but often incur high inference latency due to multi-step execution and repeated model invocations, severely limiting their scalability and usability in time-sensitive scenarios. Most existing approaches primarily optimize task performance and inference cost, and explicitly or implicitly assume sequential execution, making them less optimal for controlling latency under parallel execution. In this work, we investigate learning-based orchestration of multi-agent systems with explicit latency supervision under parallel execution. We propose Latency-Aware Multi-agent System (LAMaS), a latency-aware multi-agent orchestration framework that enables parallel execution and explicitly optimizes the critical execution path, allowing the controller to construct execution topology graphs with lower latency under parallel execution. Our experiments show that our approach reduces critical path length by 38-46% compared to the state-of-the-art baseline for multi-agent architecture search across multiple benchmarks, while maintaining or even improving task performance. These results highlight the importance of explicitly optimizing latency under parallel execution when designing efficient multi-agent systems. The code is available at https://github.com/xishi404/LAMaS
Paper Structure (30 sections, 9 equations, 3 figures, 5 tables)

This paper contains 30 sections, 9 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Strictly Sequential Orchestration.
  3. DAG-based Architectures with Cost-Centric Optimization.
  4. Static Coarse-grained Parallelism.
  5. Related work
  6. LLM-Based Multi-Agent Systems
  7. Automated Design of Agentic Systems
  8. Methodology
  9. Definitions and Parallel Execution
  10. Problem Setting
  11. Latency Modeling
  12. Controller Backbone
  13. Reward Design
  14. Critical-Path-Aware Credit Assignment
  15. Learning Objective and Normalization
  16. ...and 15 more sections

Figures (3)

  • Figure 1: (Left):Building blocks for LAMaS; (Right):Workflow illustration of LAMaS. The orchestrator generates a layer-wise execution graph, where operators within the same layer execute in parallel. Red arrows indicate the critical execution path.
  • Figure 2: Accuracy–latency trade-off on HumanEval. Marker size indicates average cost. Blue points correspond to LAMaS under different latency penalty coefficient $\lambda_t$
  • Figure 3: Case study. Red arrows highlight the critical execution path, formed by the slowest operator at each layer.