Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Monitoring LLM-based Multi-Agent Systems Against Corruptions via Node Evaluation

Chengcan Wu, Zhixin Zhang, Mingqian Xu, Zeming Wei, Meng Sun

TL;DR

This work addresses trustworthiness in LLM-based multi-agent systems by modeling inter-agent communications as a signed, time-expanded graph and applying backpropagation to quantify each agent's and edge's contribution to the final decision. It introduces a dynamic defense that identifies latent malicious agents via backward contribution scores and removes their communications to disrupt attack propagation. Across diverse datasets and models, the approach outperforms static defenses, achieving higher detection accuracy and robustness against multiple attack types, including evolving dynamic scenarios. The results demonstrate the practical potential of topology-aware, backpropagation-based monitoring for securing MAS deployments in real-world, dynamic environments.

Abstract

Large Language Model (LLM)-based Multi-Agent Systems (MAS) have become a popular paradigm of AI applications. However, trustworthiness issues in MAS remain a critical concern. Unlike challenges in single-agent systems, MAS involve more complex communication processes, making them susceptible to corruption attacks. To mitigate this issue, several defense mechanisms have been developed based on the graph representation of MAS, where agents represent nodes and communications form edges. Nevertheless, these methods predominantly focus on static graph defense, attempting to either detect attacks in a fixed graph structure or optimize a static topology with certain defensive capabilities. To address this limitation, we propose a dynamic defense paradigm for MAS graph structures, which continuously monitors communication within the MAS graph, then dynamically adjusts the graph topology, accurately disrupts malicious communications, and effectively defends against evolving and diverse dynamic attacks. Experimental results in increasingly complex and dynamic MAS environments demonstrate that our method significantly outperforms existing MAS defense mechanisms, contributing an effective guardrail for their trustworthy applications. Our code is available at https://github.com/ChengcanWu/Monitoring-LLM-Based-Multi-Agent-Systems.

Monitoring LLM-based Multi-Agent Systems Against Corruptions via Node Evaluation

TL;DR

This work addresses trustworthiness in LLM-based multi-agent systems by modeling inter-agent communications as a signed, time-expanded graph and applying backpropagation to quantify each agent's and edge's contribution to the final decision. It introduces a dynamic defense that identifies latent malicious agents via backward contribution scores and removes their communications to disrupt attack propagation. Across diverse datasets and models, the approach outperforms static defenses, achieving higher detection accuracy and robustness against multiple attack types, including evolving dynamic scenarios. The results demonstrate the practical potential of topology-aware, backpropagation-based monitoring for securing MAS deployments in real-world, dynamic environments.

Abstract

Large Language Model (LLM)-based Multi-Agent Systems (MAS) have become a popular paradigm of AI applications. However, trustworthiness issues in MAS remain a critical concern. Unlike challenges in single-agent systems, MAS involve more complex communication processes, making them susceptible to corruption attacks. To mitigate this issue, several defense mechanisms have been developed based on the graph representation of MAS, where agents represent nodes and communications form edges. Nevertheless, these methods predominantly focus on static graph defense, attempting to either detect attacks in a fixed graph structure or optimize a static topology with certain defensive capabilities. To address this limitation, we propose a dynamic defense paradigm for MAS graph structures, which continuously monitors communication within the MAS graph, then dynamically adjusts the graph topology, accurately disrupts malicious communications, and effectively defends against evolving and diverse dynamic attacks. Experimental results in increasingly complex and dynamic MAS environments demonstrate that our method significantly outperforms existing MAS defense mechanisms, contributing an effective guardrail for their trustworthy applications. Our code is available at https://github.com/ChengcanWu/Monitoring-LLM-Based-Multi-Agent-Systems.
Paper Structure (18 sections, 2 equations, 1 figure, 14 tables)

This paper contains 18 sections, 2 equations, 1 figure, 14 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. MAS Graph
  5. Extract connections
  6. Determine node contribution
  7. Experiment
  8. Experiment Set-up
  9. Result at Fixed diagram structure
  10. Dynamic Graph Experiment
  11. Ablation Study
  12. Conclusion
  13. More Details on Experiments
  14. Baseline Details
  15. Prompt in our Experiment
  16. ...and 3 more sections

Figures (1)

  • Figure 1: An overview of our method. In step 1, we reconstruct the MAS as a directed acyclic graph (DAG). In steps 2 and 3, we extract the contribution of each agent to the final decision using the contribution score on each edge and backward propagation from the final decision. This helps determine the latent malicious agents. We then repair the MAS by removing information sent from the detected malicious agents in step 4. The dashed line indicates that the communication edge has been deleted.