From Lived Experience to Insight: Unpacking the Psychological Risks of Using AI Conversational Agents
Mohit Chandra, Suchismita Naik, Denae Ford, Ebele Okoli, Munmun De Choudhury, Mahsa Ershadi, Gonzalo Ramos, Javier Hernandez, Ananya Bhattacharjee, Shahed Warreth, Jina Suh
TL;DR
This paper tackles the psychological risks of AI conversational agents by grounding taxonomy in lived experiences. It uses a two-phase mixed-methods approach: a survey with $N=283$ participants to develop a Psychological Risk Taxonomy (comprising AI Behavior, Negative Psychological Impact, and User Context) and a Phase 2 workshop with seven lived-experience experts to refine design implications via a novel multi-path vignette framework. The core contributions are a 19-behavior, 21-impact, and 15-context taxonomy, a multi-path vignette method to illustrate complex interaction pathways, and practical design recommendations to help policymakers, researchers, and developers build safer, more empathetic AI agents. The work highlights the importance of context, temporality, and severity in assessing risks and proposes actionable guidance for safer mental-health–related AI deployments, while acknowledging limitations such as sample size and scope for future expansion.
Abstract
Recent gains in popularity of AI conversational agents have led to their increased use for improving productivity and supporting well-being. While previous research has aimed to understand the risks associated with interactions with AI conversational agents, these studies often fall short in capturing the lived experiences of individuals. Additionally, psychological risks have often been presented as a sub-category within broader AI-related risks in past taxonomy works, leading to under-representation of the impact of psychological risks of AI use. To address these challenges, our work presents a novel risk taxonomy focusing on psychological risks of using AI gathered through the lived experiences of individuals. We employed a mixed-method approach, involving a comprehensive survey with 283 people with lived mental health experience and workshops involving experts with lived experience to develop a psychological risk taxonomy. Our taxonomy features 19 AI behaviors, 21 negative psychological impacts, and 15 contexts related to individuals. Additionally, we propose a novel multi-path vignette-based framework for understanding the complex interplay between AI behaviors, psychological impacts, and individual user contexts. Finally, based on the feedback obtained from the workshop sessions, we present design recommendations for developing safer and more robust AI agents. Our work offers an in-depth understanding of the psychological risks associated with AI conversational agents and provides actionable recommendations for policymakers, researchers, and developers.