DynTaskMAS: A Dynamic Task Graph-driven Framework for Asynchronous and Parallel LLM-based Multi-Agent Systems

TL;DR

DynTaskMAS tackles the inefficiencies of resource management and coordination in LLM-based multi-agent systems by introducing a dynamic task-graph-driven framework. It integrates four components—Dynamic Task Graph Generator (DTGG), Asynchronous Parallel Execution Engine (APEE), Semantic-Aware Context Management System (SACMS), and Adaptive Workflow Manager (AWM)—to enable flexible task decomposition, parallel execution, context-aware information sharing, and adaptive optimization. Empirical results show substantial improvements in execution time (21.3%–33.0%), resource utilization (35.4%), and throughput scalability up to 16 agents (3.47× for 4× agents), validating the approach across simple to complex tasks. The work provides a foundation for scalable, high-performance LLM-based MAS capable of handling dynamic, interdependent tasks efficiently.

Abstract

The emergence of Large Language Models (LLMs) in Multi-Agent Systems (MAS) has opened new possibilities for artificial intelligence, yet current implementations face significant challenges in resource management, task coordination, and system efficiency. While existing frameworks demonstrate the potential of LLM-based agents in collaborative problem-solving, they often lack sophisticated mechanisms for parallel execution and dynamic task management. This paper introduces DynTaskMAS, a novel framework that orchestrates asynchronous and parallel operations in LLM-based MAS through dynamic task graphs. The framework features four key innovations: (1) a Dynamic Task Graph Generator that intelligently decomposes complex tasks while maintaining logical dependencies, (2) an Asynchronous Parallel Execution Engine that optimizes resource utilization through efficient task scheduling, (3) a Semantic-Aware Context Management System that enables efficient information sharing among agents, and (4) an Adaptive Workflow Manager that dynamically optimizes system performance. Experimental evaluations demonstrate that DynTaskMAS achieves significant improvements over traditional approaches: a 21-33% reduction in execution time across task complexities (with higher gains for more complex tasks), a 35.4% improvement in resource utilization (from 65% to 88%), and near-linear throughput scaling up to 16 concurrent agents (3.47X improvement for 4X agents). Our framework establishes a foundation for building scalable, high-performance LLM-based multi-agent systems capable of handling complex, dynamic tasks efficiently.

Figures (3)

• Figure 1: The Overview of DynTaskMAS Framework. The diagram depicts the architecture of the DynTaskMAS framework as a multi-layered structure characterized by bidirectional information flow. At the top, the "Input Task" node serves as the entry point for complex tasks. Below it, DTGG acts as a core component, continuously analyzing and updating task graphs, with arrows connecting to the input task to signify this iterative process. The third layer comprises two parallel modules: APEE and SACMS. Both are linked to DTGG above and AWM below, reflecting their interdependent functions. Positioned in the fourth layer, AWM maintains bidirectional connections with all upper components, underscoring its role in managing and optimizing workflows. At the base, multiple "LLM-based Agent" nodes are connected to APEE and SACMS, illustrating the distributed execution of tasks and continuous exchange of information.
• Figure 2: The architecture of Semantic-Aware Context Management System. The SACMS architecture comprises five core components: (1) the Context Repository, a distributed, hierarchical data store designed for efficient storage and retrieval of contextual information; (2) the Semantic Analyzer, which employs advanced natural language processing and domain-specific ontologies to extract meaningful semantic tags and relationships; (3) the Context Distribution Manager, responsible for the efficient dissemination of relevant contextual information to agents based on task requirements and semantic relevance; (4) the Query Processor, which utilizes semantic matching to process context retrieval requests and deliver the most pertinent information; and (5) the Update Handler, which integrates new or updated context data into the repository while maintaining the semantic index. Together, these components enable intelligent, context-aware task execution within the DynTaskMAS system.
• Figure 3: The comparison between traditional processing and DynTaskMAS framework. The system was deployed on an NVIDIA RTX 3090 GPU with Llama-3.1-8B serving as the foundation model for all agents. Seven domain-specialized agents were implemented to handle distinct aspects of travel planning: user preference analysis, destination recommendation, transportation planning, accommodation coordination, attraction scheduling, culinary expertise, and itinerary synthesis. The experimental results showed that DynTaskMAS (3.7s) achieved faster execution compared to serial execution (4.7s).