Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PRISON: Unmasking the Criminal Potential of Large Language Models

Xinyi Wu, Geng Hong, Pei Chen, Yueyue Chen, Xudong Pan, Min Yang

TL;DR

This paper tackles safety risks of large language models in realistic social settings by introducing PRISON, a tri-perspective framework that quantifies criminal potential and detection capability via five defined traits. It leverages film-inspired scenarios rewritten to prevent memorization and uses three perspectives—Criminal, Detective, and God—to measure trait activation (CTAR) and detection accuracy (OTDA) in adversarial interactions. Empirical results show that state-of-the-art LLMs frequently emit criminal traits (CTAR > 0.5) even without explicit prompts, while detection capabilities lag (average OTDA ~0.44), revealing a concerning mismatch that may amplify misuse risks. The findings motivate stronger adversarial robustness, behavioral alignment, and governance to ensure safer broad deployment, and the authors offer the scenario dataset and framework to support ongoing safety research.

Abstract

As large language models (LLMs) advance, concerns about their misconduct in complex social contexts intensify. Existing research overlooked the systematic understanding and assessment of their criminal capability in realistic interactions. We propose a unified framework PRISON, to quantify LLMs' criminal potential across five traits: False Statements, Frame-Up, Psychological Manipulation, Emotional Disguise, and Moral Disengagement. Using structured crime scenarios adapted from classic films grounded in reality, we evaluate both criminal potential and anti-crime ability of LLMs. Results show that state-of-the-art LLMs frequently exhibit emergent criminal tendencies, such as proposing misleading statements or evasion tactics, even without explicit instructions. Moreover, when placed in a detective role, models recognize deceptive behavior with only 44% accuracy on average, revealing a striking mismatch between conducting and detecting criminal behavior. These findings underscore the urgent need for adversarial robustness, behavioral alignment, and safety mechanisms before broader LLM deployment.

PRISON: Unmasking the Criminal Potential of Large Language Models

TL;DR

This paper tackles safety risks of large language models in realistic social settings by introducing PRISON, a tri-perspective framework that quantifies criminal potential and detection capability via five defined traits. It leverages film-inspired scenarios rewritten to prevent memorization and uses three perspectives—Criminal, Detective, and God—to measure trait activation (CTAR) and detection accuracy (OTDA) in adversarial interactions. Empirical results show that state-of-the-art LLMs frequently emit criminal traits (CTAR > 0.5) even without explicit prompts, while detection capabilities lag (average OTDA ~0.44), revealing a concerning mismatch that may amplify misuse risks. The findings motivate stronger adversarial robustness, behavioral alignment, and governance to ensure safer broad deployment, and the authors offer the scenario dataset and framework to support ongoing safety research.

Abstract

As large language models (LLMs) advance, concerns about their misconduct in complex social contexts intensify. Existing research overlooked the systematic understanding and assessment of their criminal capability in realistic interactions. We propose a unified framework PRISON, to quantify LLMs' criminal potential across five traits: False Statements, Frame-Up, Psychological Manipulation, Emotional Disguise, and Moral Disengagement. Using structured crime scenarios adapted from classic films grounded in reality, we evaluate both criminal potential and anti-crime ability of LLMs. Results show that state-of-the-art LLMs frequently exhibit emergent criminal tendencies, such as proposing misleading statements or evasion tactics, even without explicit instructions. Moreover, when placed in a detective role, models recognize deceptive behavior with only 44% accuracy on average, revealing a striking mismatch between conducting and detecting criminal behavior. These findings underscore the urgent need for adversarial robustness, behavioral alignment, and safety mechanisms before broader LLM deployment.

Paper Structure

This paper contains 51 sections, 12 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. PRISON: Perspective Recognition In Statement ObservatioN
  4. Criminal Traits
  5. Three Perspective Recognition
  6. Scenario Construction
  7. Experiment 1: Revealing LLMs' Criminal Potential
  8. Experiment Setup
  9. Results
  10. Experiment 2: Assessing LLMs' Crime Detection Capability
  11. Experiment Setup
  12. Results
  13. Discussion
  14. Implications.
  15. Future Work.
  16. ...and 36 more sections

Figures (7)

  • Figure 1: Framework for Evaluating Criminal Potential and Detection Capability Based on Perspective Recognition in Statement Observation (Three Perspectives: Criminal, Detective and God)
  • Figure 2: A simplified Scenario Example
  • Figure 3: Criminal Traits Activation Rate with and without Instruction in Different LLMs
  • Figure 4: Criminal Traits Activation Rate across Dialogue Turns with and without Instruction
  • Figure 5: Each Criminal Trait Activation Rates with and without Instruction
  • ...and 2 more figures