PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety

TL;DR

PsySafe introduces a psychology-grounded framework to study and mitigate safety risks in multi-agent systems powered by LLMs. It systematically combines dark-traits–based attacks, dual safety evaluation (psychological and behavioral metrics like Process Danger Rate and Joint Danger Rate), and defense mechanisms (input, Doctor, and Police defenses). The study reveals that dark personality injections elevate collective dangerous behaviors, that psychological scores correlate with behavioral risk, and that self-reflection across rounds can reduce danger, with defense strategies further curbing risk. The work provides empirical insights across multiple MAS platforms and LLMs, highlights limitations of current defenses, and releases data and code to support future safety research in MAS.

Abstract

Multi-agent systems, when enhanced with Large Language Models (LLMs), exhibit profound capabilities in collective intelligence. However, the potential misuse of this intelligence for malicious purposes presents significant risks. To date, comprehensive research on the safety issues associated with multi-agent systems remains limited. In this paper, we explore these concerns through the innovative lens of agent psychology, revealing that the dark psychological states of agents constitute a significant threat to safety. To tackle these concerns, we propose a comprehensive framework (PsySafe) grounded in agent psychology, focusing on three key areas: firstly, identifying how dark personality traits in agents can lead to risky behaviors; secondly, evaluating the safety of multi-agent systems from the psychological and behavioral perspectives, and thirdly, devising effective strategies to mitigate these risks. Our experiments reveal several intriguing phenomena, such as the collective dangerous behaviors among agents, agents' self-reflection when engaging in dangerous behavior, and the correlation between agents' psychological assessments and dangerous behaviors. We anticipate that our framework and observations will provide valuable insights for further research into the safety of multi-agent systems. We will make our data and code publicly accessible at https://github.com/AI4Good24/PsySafe.

Figures (20)

• Figure 1: Examples of Agents' Interactions after Psychological-based Attack. After being attacked, the multi-agent system, whether for safe daily tasks or dangerous jailbreak tasks, provides dangerous answers. Agents collaborate with each other to generate dangerous content. Responses identified as dangerous are highlighted in red, whereas safe responses are indicated in green.
• Figure 2: Overview of PsySafe. 'Psychology' denotes the six moral dimensions we adopt. 'Attack' refers to our attack methodology, including the construction of attack prompts and the exploration of various angles in attacking multi-agent systems. 'Agent System' refers to the prevalent frameworks among current multi-agent systems, comprising hierarchical, joint, and hybrid structures. 'Defense' signifies the defensive strategies we propose, encompassing input, doctor, and police defense mechanisms. 'Evaluation' represents our evaluation techniques, encompassing psychological evaluation and the identification of joint and process danger conditions.
• Figure 3: Doctor Defense. Doctor defense strategy encompasses two primary components. Initially, Psychological evaluations are conducted for all agents within a multi-agent system. Based on the evaluation scores, contaminated agents are identified. The results of these psychological assessments, along with the agents' system prompts, are then forwarded to the doctor agent. In response, the doctor agent generates a new system prompt specifically tailored for the contaminated agents. Subsequently, both the evaluation and this entire process are repeated until the psychological scores reach the pre-defined standards $P$, which, in our experiments, is set as 20.
• Figure 4: Joint Danger Rates across Different Rounds. The joint danger rates exhibit a declining trend with the increase in the number of rounds for Camel and AutoGen.
• Figure 5: Distributions of Psychological Test Scores for Safe (Blue) and Dangerous (Red) Behaviors Across Four Multi-agent Systems. This figure shows the distribution between agents' psychological test scores and the safety of their behaviors, indicating a general trend where agents with more dangerous scores are more likely to exhibit dangerous behaviors.