Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AgentDropout: Dynamic Agent Elimination for Token-Efficient and High-Performance LLM-Based Multi-Agent Collaboration

Zhexuan Wang, Yutong Wang, Xuebo Liu, Liang Ding, Miao Zhang, Jie Liu, Min Zhang

TL;DR

This work addresses token inefficiency in LLM-based multi-agent collaboration by proposing AgentDropout, a dynamic topology optimization method that jointly prunes redundant agent roles and communication edges across rounds. It learns weighted intra- and inter-round adjacency matrices and uses DAG sampling to realize a compact, effective multi-round graph, improving both task performance and token efficiency. Extensive experiments across multiple models and tasks demonstrate superior accuracy, stability, and significant reductions in prompt and completion tokens, with strong domain transferability and robustness. The approach offers a practical framework for scalable, efficient MAS and highlights the value of adaptive role assignments in complex collaborative reasoning.

Abstract

Multi-agent systems (MAS) based on large language models (LLMs) have demonstrated significant potential in collaborative problem-solving. However, they still face substantial challenges of low communication efficiency and suboptimal task performance, making the careful design of the agents' communication topologies particularly important. Inspired by the management theory that roles in an efficient team are often dynamically adjusted, we propose AgentDropout, which identifies redundant agents and communication across different communication rounds by optimizing the adjacency matrices of the communication graphs and eliminates them to enhance both token efficiency and task performance. Compared to state-of-the-art methods, AgentDropout achieves an average reduction of 21.6% in prompt token consumption and 18.4% in completion token consumption, along with a performance improvement of 1.14 on the tasks. Furthermore, the extended experiments demonstrate that AgentDropout achieves notable domain transferability and structure robustness, revealing its reliability and effectiveness. We release our code at https://github.com/wangzx1219/AgentDropout.

AgentDropout: Dynamic Agent Elimination for Token-Efficient and High-Performance LLM-Based Multi-Agent Collaboration

TL;DR

This work addresses token inefficiency in LLM-based multi-agent collaboration by proposing AgentDropout, a dynamic topology optimization method that jointly prunes redundant agent roles and communication edges across rounds. It learns weighted intra- and inter-round adjacency matrices and uses DAG sampling to realize a compact, effective multi-round graph, improving both task performance and token efficiency. Extensive experiments across multiple models and tasks demonstrate superior accuracy, stability, and significant reductions in prompt and completion tokens, with strong domain transferability and robustness. The approach offers a practical framework for scalable, efficient MAS and highlights the value of adaptive role assignments in complex collaborative reasoning.

Abstract

Multi-agent systems (MAS) based on large language models (LLMs) have demonstrated significant potential in collaborative problem-solving. However, they still face substantial challenges of low communication efficiency and suboptimal task performance, making the careful design of the agents' communication topologies particularly important. Inspired by the management theory that roles in an efficient team are often dynamically adjusted, we propose AgentDropout, which identifies redundant agents and communication across different communication rounds by optimizing the adjacency matrices of the communication graphs and eliminates them to enhance both token efficiency and task performance. Compared to state-of-the-art methods, AgentDropout achieves an average reduction of 21.6% in prompt token consumption and 18.4% in completion token consumption, along with a performance improvement of 1.14 on the tasks. Furthermore, the extended experiments demonstrate that AgentDropout achieves notable domain transferability and structure robustness, revealing its reliability and effectiveness. We release our code at https://github.com/wangzx1219/AgentDropout.

Paper Structure

This paper contains 35 sections, 8 equations, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background
  3. LLM-based MAS as Graph
  4. Communication Redundancy in MAS
  5. AgentDropout
  6. Overview
  7. Motivation
  8. Preparation
  9. Node Dropout
  10. Edge Dropout
  11. Discussion
  12. Experiments
  13. Experimental Setup
  14. Models and Benchmarks
  15. Baselines
  16. ...and 20 more sections

Figures (3)

  • Figure 1: An overview of AgentDropout, in comparison with vanilla MAS and AgentPrune zhang2025cut, highlighting its dynamic adjustments of participant roles in each discussion to enhance cooperation effectiveness and efficiency.
  • Figure 2: The overall process of AgentDropout. The first and second rows present Node Dropout and Edge Dropout procedures, respectively. The third row illustrates the cooperative reasoning process for both intra- and inter-round communication, as well as the generation of the final answer. For simplicity, the edges of inter-round communication are omitted in the demonstration of step 1 7, but they also participate in the update, dropout, and reasoning processes as shown in the middle block of the third row.
  • Figure 3: Full connected, layer and random graph as demonstrations of intra-round communication topologies.