MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systems

TL;DR

MedSentry addresses the safety risks of medical LLM multi-agent systems by introducing a large adversarial dataset and a topology-aware attack-defense evaluation framework. It benchmarks four common MAS topologies (Layers, SharedPool, Centralized, Decentralized) against 'dark-personality' insiders and reveals distinct vulnerability patterns, notably SharedPool's susceptibility and Decentralized resilience. The authors propose a lightweight, behavior-informed Enforcement Agent (PCDC) using psychometric screening, behavior verification, and topology-aware isolation to restore safety near baseline across architectures. The work provides both a rigorous evaluation protocol and practical mitigation strategies to guide the design of safer medical AI collaboration systems.

Abstract

As large language models (LLMs) are increasingly deployed in healthcare, ensuring their safety, particularly within collaborative multi-agent configurations, is paramount. In this paper we introduce MedSentry, a benchmark comprising 5 000 adversarial medical prompts spanning 25 threat categories with 100 subthemes. Coupled with this dataset, we develop an end-to-end attack-defense evaluation pipeline to systematically analyze how four representative multi-agent topologies (Layers, SharedPool, Centralized, and Decentralized) withstand attacks from 'dark-personality' agents. Our findings reveal critical differences in how these architectures handle information contamination and maintain robust decision-making, exposing their underlying vulnerability mechanisms. For instance, SharedPool's open information sharing makes it highly susceptible, whereas Decentralized architectures exhibit greater resilience thanks to inherent redundancy and isolation. To mitigate these risks, we propose a personality-scale detection and correction mechanism that identifies and rehabilitates malicious agents, restoring system safety to near-baseline levels. MedSentry thus furnishes both a rigorous evaluation framework and practical defense strategies that guide the design of safer LLM-based multi-agent systems in medical domains.

Figures (12)

• Figure 1: Overview of our two‐phase MedSentry construction pipeline. (A) shows data topic definition phase with predefined risk categories and progressive topic filters; (B) demonstrates data generation and refinement phase via template-based generation and human-AI collaborative curation.
• Figure 2: The achitecture of our proposed MedSentry evaluation workflow. (A) a safety‑critical MedSentry query is injected. (B) the query propagates through four multi-agent topologies (i.e., centralized, decentralized, layer, and shared‑pool), each embedding a single dark‑personality agent to stress-test safety and stability. (C) the enforcement agent screens/adjudicates their behaviors and isolates malicious offender if necessary, enabling systematic safety comparison across all architectures.
• Figure 3: Multi-agent system defense evaluation. (a) shows absolute scores across conditions. (b) demonstrates defense improvement over attack and comparison to baseline.
• Figure 4: Impact of debate rounds on LCS and RS across vairous topologies.
• Figure 5: Impact of agent number on LCS and RS across various topologies.