Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?

Soorya Ram Shimgekar, Vipin Gunda, Jiwon Kim, Violeta J. Rodriguez, Hari Sundaram, Koustuv Saha

Abstract

Conversational AI systems are increasingly used for personal reflection and emotional disclosure, raising concerns about their effects on vulnerable users. Recent anecdotal reports suggest that prolonged interactions with AI may reinforce delusional thinking -- a phenomenon sometimes described as AI Psychosis. However, empirical evidence on this phenomenon remains limited. In this work, we examine how delusion-related language evolves during multi-turn interactions with conversational AI. We construct simulated users (SimUsers) from Reddit users' longitudinal posting histories and generate extended conversations with three model families (GPT, LLaMA, and Qwen). We develop DelusionScore, a linguistic measure that quantifies the intensity of delusion-related language across conversational turns. We find that SimUsers derived from users with prior delusion-related discourse (Treatment) exhibit progressively increasing DelusionScore trajectories, whereas those derived from users without such discourse (Control) remain stable or decline. We further find that this amplification varies across themes, with reality skepticism and compulsive reasoning showing the strongest increases. Finally, conditioning AI responses on current DelusionScore substantially reduces these trajectories. These findings provide empirical evidence that conversational AI interactions can amplify delusion-related language over extended use and highlight the importance of state-aware safety mechanisms for mitigating such risks.

AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?

Abstract

Conversational AI systems are increasingly used for personal reflection and emotional disclosure, raising concerns about their effects on vulnerable users. Recent anecdotal reports suggest that prolonged interactions with AI may reinforce delusional thinking -- a phenomenon sometimes described as AI Psychosis. However, empirical evidence on this phenomenon remains limited. In this work, we examine how delusion-related language evolves during multi-turn interactions with conversational AI. We construct simulated users (SimUsers) from Reddit users' longitudinal posting histories and generate extended conversations with three model families (GPT, LLaMA, and Qwen). We develop DelusionScore, a linguistic measure that quantifies the intensity of delusion-related language across conversational turns. We find that SimUsers derived from users with prior delusion-related discourse (Treatment) exhibit progressively increasing DelusionScore trajectories, whereas those derived from users without such discourse (Control) remain stable or decline. We further find that this amplification varies across themes, with reality skepticism and compulsive reasoning showing the strongest increases. Finally, conditioning AI responses on current DelusionScore substantially reduces these trajectories. These findings provide empirical evidence that conversational AI interactions can amplify delusion-related language over extended use and highlight the importance of state-aware safety mechanisms for mitigating such risks.
Paper Structure (21 sections, 4 figures, 7 tables)

This paper contains 21 sections, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data
  4. Methods
  5. Matching Treatment and Control Users
  6. Simulating Multi-Turn Conversations
  7. Estimating Delusion in Multi-turn Conversations: DelusionScore
  8. Themes in Delusion-related Language
  9. DelusionScore-Conditioned Setup
  10. Findings
  11. RQ1: Evolution of Delusion-Related Language in Human-AI Conversations
  12. RQ2: Thematic Trajectories of Delusion-Related Language
  13. RQ3: Conditioning AI Responses on Delusion-Related Language
  14. Discussion and Conclusion
  15. Theoretical Implications
  16. ...and 6 more sections

Figures (4)

  • Figure 1: (a) Standardized mean difference (SMD) by varying the number of propensity score strata; (b) Distribution of SMDs, dotted vertical lines show the mean values; (c) Topic coherence ($c_v$) as a function of the number of topics.
  • Figure 2: Average score trajectories across dialogue turns for large language models, stratified by user-level propensity score bins. Curves are smoothened using LOWESS.
  • Figure :
  • Figure A1: Average score trajectories across dialogue turns for large language models, stratified by user-level propensity score bins.