Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bullying the Machine: How Personas Increase LLM Vulnerability

Ziwei Xu, Udit Sanghi, Mohan Kankanhalli

TL;DR

This work addresses whether conditioning LLMs to adopt human-like personas affects safety under adversarial bullying. It introduces a simulation framework with attacker and victim LLMs, nine bullying tactics, and Big Five personality-based victim prompts, and studies safety across multiple open-source models and two datasets (Mini-5 and AdvBench-50), with broader insights from AdvBench-520. The key findings show that certain victim traits (e.g., lower Agreeableness and Conscientiousness, higher Extroversion) increase susceptibility to unsafe outputs, while specific tactics (Gaslighting, Passive Aggression, Mocking) are particularly effective; results also reveal that longer conversations exacerbate safety risks. The paper highlights the need for persona-aware safety evaluation and dynamic guardrails in LLMs, suggesting that safety is not uniform across models or interaction contexts and that manipulation vectors could be exploited for both attacks and defenses in real-world deployments.

Abstract

Large Language Models (LLMs) are increasingly deployed in interactions where they are prompted to adopt personas. This paper investigates whether such persona conditioning affects model safety under bullying, an adversarial manipulation that applies psychological pressures in order to force the victim to comply to the attacker. We introduce a simulation framework in which an attacker LLM engages a victim LLM using psychologically grounded bullying tactics, while the victim adopts personas aligned with the Big Five personality traits. Experiments using multiple open-source LLMs and a wide range of adversarial goals reveal that certain persona configurations -- such as weakened agreeableness or conscientiousness -- significantly increase victim's susceptibility to unsafe outputs. Bullying tactics involving emotional or sarcastic manipulation, such as gaslighting and ridicule, are particularly effective. These findings suggest that persona-driven interaction introduces a novel vector for safety risks in LLMs and highlight the need for persona-aware safety evaluation and alignment strategies.

Bullying the Machine: How Personas Increase LLM Vulnerability

TL;DR

This work addresses whether conditioning LLMs to adopt human-like personas affects safety under adversarial bullying. It introduces a simulation framework with attacker and victim LLMs, nine bullying tactics, and Big Five personality-based victim prompts, and studies safety across multiple open-source models and two datasets (Mini-5 and AdvBench-50), with broader insights from AdvBench-520. The key findings show that certain victim traits (e.g., lower Agreeableness and Conscientiousness, higher Extroversion) increase susceptibility to unsafe outputs, while specific tactics (Gaslighting, Passive Aggression, Mocking) are particularly effective; results also reveal that longer conversations exacerbate safety risks. The paper highlights the need for persona-aware safety evaluation and dynamic guardrails in LLMs, suggesting that safety is not uniform across models or interaction contexts and that manipulation vectors could be exploited for both attacks and defenses in real-world deployments.

Abstract

Large Language Models (LLMs) are increasingly deployed in interactions where they are prompted to adopt personas. This paper investigates whether such persona conditioning affects model safety under bullying, an adversarial manipulation that applies psychological pressures in order to force the victim to comply to the attacker. We introduce a simulation framework in which an attacker LLM engages a victim LLM using psychologically grounded bullying tactics, while the victim adopts personas aligned with the Big Five personality traits. Experiments using multiple open-source LLMs and a wide range of adversarial goals reveal that certain persona configurations -- such as weakened agreeableness or conscientiousness -- significantly increase victim's susceptibility to unsafe outputs. Bullying tactics involving emotional or sarcastic manipulation, such as gaslighting and ridicule, are particularly effective. These findings suggest that persona-driven interaction introduces a novel vector for safety risks in LLMs and highlight the need for persona-aware safety evaluation and alignment strategies.
Paper Structure (35 sections, 2 equations, 5 figures, 5 tables)

This paper contains 35 sections, 2 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Psychological Traits in LLMs
  4. Prompt-Based Attacks on LLMs
  5. Characteristics and Dynamics of Bullying
  6. Automatic Conversation Simulation
  7. Formulation
  8. Prompting LLMs with Personas, Goals, and Tactics
  9. A Conversation and its Safety
  10. Victim Configuration
  11. Attacker Configuration
  12. Conversation Generation and Safety Evaluation
  13. Experiments
  14. Dataset
  15. Implementation
  16. ...and 20 more sections

Figures (5)

  • Figure 1: The simulation of an example dialogue between an attacker and a victim LLM.
  • Figure 2: Unsafe@5 by victim's persona on Mini-5 and AdvBench-50. Base indicates that no modification is applied to the victim models. A "$\uparrow$" indicates a strengthened BF dimension, and a "$\downarrow$" indicates a weakened one. Full form of the BF dimensions are listed in \ref{['tab:victim_personalities']} Standard deviation is computed based on five and three runs, on Mini-5 and AdvBench-50, respectively.
  • Figure 3: Unsafe@5 by attacker's tactic on Mini-5 and AdvBench-50. AG: aggression, GL: gaslighting, MP: guilt tripping, PA: passive aggression, MR: mocking and ridicule, AI: authority intimidation, RP: repetitive pressure, TC: threatening coercion. Details of the tactics are listed in \ref{['tab:attack_tactics']}. Standard deviation is computed based on five and three runs, on Mini-5 and AdvBench-50, respectively.
  • Figure 4: Unsafe@$k$, where $k=1,2,\ldots,5$ is round of conversation.
  • Figure E5: The change of unsafe@5 rate caused by various personas and tactics on Mini-5 averaged over all the LLMs (Llama-3.1-8B, Mistral-7B, Qwen-2.5-14B, and Qwen-3-32B), compared with Base persona and Base tactic. The horizontal axis lists the victim's personas. The vertical axis lists the attacker's tactics. The numbers is the absolute difference in the unsafe@5 rates (%).