Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exposing Weak Links in Multi-Agent Systems under Adversarial Prompting

Nirmit Arora, Sathvik Joel, Ishan Kavathekar, Palak, Rohan Gandhi, Yash Pandya, Tanuja Ganu, Aditya Kanade, Akshay Nambi

TL;DR

The paper addresses the vulnerability of multi-agent systems (MAS) to adversarial prompting, highlighting that existing single-agent safety guarantees do not easily transfer to MAS. It introduces SafeAgents, a unified framework for fine-grained MAS security analysis, and Dharma, a design-aware diagnostic metric that localizes failure modes within MAS pipelines. Through extensive experiments across five MAS architectures and four safety benchmarks, the study reveals that common design patterns can create weak links, such as centralized orchestration that masks harmful objectives or fails to deploy effective fallbacks. The work demonstrates the need for architecture-aware security in MAS and provides actionable tooling to guide robust MAS design and evaluation.

Abstract

LLM-based agents are increasingly deployed in multi-agent systems (MAS). As these systems move toward real-world applications, their security becomes paramount. Existing research largely evaluates single-agent security, leaving a critical gap in understanding the vulnerabilities introduced by multi-agent design. However, existing systems fall short due to lack of unified frameworks and metrics focusing on unique rejection modes in MAS. We present SafeAgents, a unified and extensible framework for fine-grained security assessment of MAS. SafeAgents systematically exposes how design choices such as plan construction strategies, inter-agent context sharing, and fallback behaviors affect susceptibility to adversarial prompting. We introduce Dharma, a diagnostic measure that helps identify weak links within multi-agent pipelines. Using SafeAgents, we conduct a comprehensive study across five widely adopted multi-agent architectures (centralized, decentralized, and hybrid variants) on four datasets spanning web tasks, tool use, and code generation. Our findings reveal that common design patterns carry significant vulnerabilities. For example, centralized systems that delegate only atomic instructions to sub-agents obscure harmful objectives, reducing robustness. Our results highlight the need for security-aware design in MAS. Link to code is https://github.com/microsoft/SafeAgents

Exposing Weak Links in Multi-Agent Systems under Adversarial Prompting

TL;DR

The paper addresses the vulnerability of multi-agent systems (MAS) to adversarial prompting, highlighting that existing single-agent safety guarantees do not easily transfer to MAS. It introduces SafeAgents, a unified framework for fine-grained MAS security analysis, and Dharma, a design-aware diagnostic metric that localizes failure modes within MAS pipelines. Through extensive experiments across five MAS architectures and four safety benchmarks, the study reveals that common design patterns can create weak links, such as centralized orchestration that masks harmful objectives or fails to deploy effective fallbacks. The work demonstrates the need for architecture-aware security in MAS and provides actionable tooling to guide robust MAS design and evaluation.

Abstract

LLM-based agents are increasingly deployed in multi-agent systems (MAS). As these systems move toward real-world applications, their security becomes paramount. Existing research largely evaluates single-agent security, leaving a critical gap in understanding the vulnerabilities introduced by multi-agent design. However, existing systems fall short due to lack of unified frameworks and metrics focusing on unique rejection modes in MAS. We present SafeAgents, a unified and extensible framework for fine-grained security assessment of MAS. SafeAgents systematically exposes how design choices such as plan construction strategies, inter-agent context sharing, and fallback behaviors affect susceptibility to adversarial prompting. We introduce Dharma, a diagnostic measure that helps identify weak links within multi-agent pipelines. Using SafeAgents, we conduct a comprehensive study across five widely adopted multi-agent architectures (centralized, decentralized, and hybrid variants) on four datasets spanning web tasks, tool use, and code generation. Our findings reveal that common design patterns carry significant vulnerabilities. For example, centralized systems that delegate only atomic instructions to sub-agents obscure harmful objectives, reducing robustness. Our results highlight the need for security-aware design in MAS. Link to code is https://github.com/microsoft/SafeAgents

Paper Structure

This paper contains 20 sections, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Architectural Variants
  5. Adversarial Prompting Benchmarks
  6. Evaluation Metrics
  7. Challenges
  8. Methodology
  9. Abstracting Design Primitives
  10. DHARMA: evaluation metric
  11. Implementation and extensibility of SafeAgents
  12. Analysis Results
  13. How do different architectures compare to each other in terms of safety?
  14. What are the weak links in Centralized Architecture?
  15. Impact of SLM
  16. ...and 5 more sections

Figures (6)

  • Figure 1: Orchestrator (left) and its subagents (right) in the Magentic setup.
  • Figure 2: Orchestrator (left) and its subagents (right) in the LangGraph setup.
  • Figure 3: Dharma Classification Flowchart: Decision tree showing how execution trajectories are classified based on planner and subagent behavior. Each path represents a different execution outcome in MAS and resulting Dharma class
  • Figure 4: Magentic centralized agent invokes the tool user agent on one of the tasks of AgentHarm despite understanding that it is a harmful task. This is because of absence of clear instructions on what to do when there is a failure in plan generation phase
  • Figure 5: Magentic centralized agent invokes web surfer, giving it atomic instructions. As a result, it would never refust to do the task
  • ...and 1 more figures