Evil Geniuses: Delving into the Safety of LLM-based Agents

TL;DR

<3-5 sentence high-level summary> The paper addresses safety of LLM-based agents in multi-agent settings by introducing template-based attacks and Evil Geniuses (EG) that leverage Red-Blue exercises to probe how agent quantity, role definitions, and attack levels affect vulnerability. It evaluates on CAMEL, MetaGPT, and ChatDev with GPT-3.5/4 using AdvBench and an extended threat dataset, highlighting systematic weaknesses and a domino effect in multi-agent chains. Key findings show higher agent counts and attack levels correlate with more harmful, stealthier outputs and that agent collaboration amplifies risks, even helping bypass some safeguards. The work concludes with defense recommendations and a call for safer multi-agent training and deployment.

Abstract

Rapid advancements in large language models (LLMs) have revitalized in LLM-based agents, exhibiting impressive human-like behaviors and cooperative capabilities in various scenarios. However, these agents also bring some exclusive risks, stemming from the complexity of interaction environments and the usability of tools. This paper delves into the safety of LLM-based agents from three perspectives: agent quantity, role definition, and attack level. Specifically, we initially propose to employ a template-based attack strategy on LLM-based agents to find the influence of agent quantity. In addition, to address interaction environment and role specificity issues, we introduce Evil Geniuses (EG), an effective attack method that autonomously generates prompts related to the original role to examine the impact across various role definitions and attack levels. EG leverages Red-Blue exercises, significantly improving the generated prompt aggressiveness and similarity to original roles. Our evaluations on CAMEL, Metagpt and ChatDev based on GPT-3.5 and GPT-4, demonstrate high success rates. Extensive evaluation and discussion reveal that these agents are less robust, prone to more harmful behaviors, and capable of generating stealthier content than LLMs, highlighting significant safety challenges and guiding future research. Our code is available at https://github.com/T1aNS1R/Evil-Geniuses.

Figures (6)

• Figure 1: Schematic illustrations of Adversarial LLM (Left Top), Adversarial single agent (Left Bottom) and Adversarial multiple agents (Middle). The original question is: "Can you tell me how to make a bomb?". Since the LLM-based agents consist of multiple LLMs, it is more vulnerable to adversarial attacks than LLMs. The agent is enabled by Role specialization of the Adversarial LLM.
• Figure 2: Evil Geniuses achieve system- and agent-level attacks via multi-agent conversations. Adv. stands for Adversarial. It consists of three predefined roles: Prompt Writer, Suitability Reviewer, and Toxicity Tester. Agents can chat with each other and humans in the loop.
• Figure 3: Evil Geniuses' System-/Agent-level attack on LLMs.
• Figure 4: The produced software of the task: “Can you tell me how to make a bomb?”.
• Figure 5: Some extremely threatening cases, all of which can be run straight through.