Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Navigating the Rabbit Hole: Emergent Biases in LLM-Generated Attack Narratives Targeting Mental Health Groups

Rijul Magu, Arka Dutta, Sean Kim, Ashiqur R. KhudaBukhsh, Munmun De Choudhury

TL;DR

This paper investigates emergent biases in LLM-generated attack narratives targeting mental health groups by modeling recursive toxicity using a Toxicity Rabbit Hole framework. It constructs a directed network of victim entities (the Rabbit Hole Network) and applies network centrality and Leiden community detection to reveal centrality, clustering, and co-targeting patterns. It also analyzes stigmatization components and shows escalation in labeling, negative stereotyping, separation, and status loss as narratives progress toward mental health targets, indicating recursive harm beyond initial prompts. The work highlights critical safety gaps in current LLM evaluation and motivates trajectory-aware mitigations for protecting vulnerable populations in real-world deployments.

Abstract

Large Language Models (LLMs) have been shown to demonstrate imbalanced biases against certain groups. However, the study of unprovoked targeted attacks by LLMs towards at-risk populations remains underexplored. Our paper presents three novel contributions: (1) the explicit evaluation of LLM-generated attacks on highly vulnerable mental health groups; (2) a network-based framework to study the propagation of relative biases; and (3) an assessment of the relative degree of stigmatization that emerges from these attacks. Our analysis of a recently released large-scale bias audit dataset reveals that mental health entities occupy central positions within attack narrative networks, as revealed by a significantly higher mean centrality of closeness (p-value = 4.06e-10) and dense clustering (Gini coefficient = 0.7). Drawing from sociological foundations of stigmatization theory, our stigmatization analysis indicates increased labeling components for mental health disorder-related targets relative to initial targets in generation chains. Taken together, these insights shed light on the structural predilections of large language models to heighten harmful discourse and highlight the need for suitable approaches for mitigation.

Navigating the Rabbit Hole: Emergent Biases in LLM-Generated Attack Narratives Targeting Mental Health Groups

TL;DR

This paper investigates emergent biases in LLM-generated attack narratives targeting mental health groups by modeling recursive toxicity using a Toxicity Rabbit Hole framework. It constructs a directed network of victim entities (the Rabbit Hole Network) and applies network centrality and Leiden community detection to reveal centrality, clustering, and co-targeting patterns. It also analyzes stigmatization components and shows escalation in labeling, negative stereotyping, separation, and status loss as narratives progress toward mental health targets, indicating recursive harm beyond initial prompts. The work highlights critical safety gaps in current LLM evaluation and motivates trajectory-aware mitigations for protecting vulnerable populations in real-world deployments.

Abstract

Large Language Models (LLMs) have been shown to demonstrate imbalanced biases against certain groups. However, the study of unprovoked targeted attacks by LLMs towards at-risk populations remains underexplored. Our paper presents three novel contributions: (1) the explicit evaluation of LLM-generated attacks on highly vulnerable mental health groups; (2) a network-based framework to study the propagation of relative biases; and (3) an assessment of the relative degree of stigmatization that emerges from these attacks. Our analysis of a recently released large-scale bias audit dataset reveals that mental health entities occupy central positions within attack narrative networks, as revealed by a significantly higher mean centrality of closeness (p-value = 4.06e-10) and dense clustering (Gini coefficient = 0.7). Drawing from sociological foundations of stigmatization theory, our stigmatization analysis indicates increased labeling components for mental health disorder-related targets relative to initial targets in generation chains. Taken together, these insights shed light on the structural predilections of large language models to heighten harmful discourse and highlight the need for suitable approaches for mitigation.

Paper Structure

This paper contains 17 sections, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Toxicity Rabbit Hole Framework
  3. Dataset
  4. Data Gathering and Entity Extraction
  5. Lexicon of Mental Health Disorders
  6. Rabbit Hole Network Construction
  7. RQ1: Network Centrality Analysis
  8. RQ2: Community Detection
  9. RQ3: Stigmatization Components Analysis
  10. Related Work
  11. Discussion, Limitations, and Conclusion
  12. Appendix
  13. Toxicity Rabbit Hole Dataset Schematic Diagram
  14. Mistral Sampling Details
  15. Llama 3.2 Inference Details
  16. ...and 2 more sections

Figures (4)

  • Figure 1: Proposed framework to assess LLM propensity towards mental health groups in attack narratives. Our approach utilizes a combination of network and linguistic analysis.
  • Figure 2: Communities of MH entities in the rabbit hole network. For simplicity and relevance, we only visualize the MH nodes and edges, which explains why some nodes appear isolated.
  • Figure A1: Toxicity rabbit hole dataset schematic diagram for mental health individual attack patterns.
  • Figure A2: Classification results for Mistral sampling