Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

"It Listens Better Than My Therapist": Exploring Social Media Discourse on LLMs as Mental Health Tool

Anna-Carolina Haensch

TL;DR

This paper investigates how people discuss using large language models (LLMs) for mental health support on TikTok. It develops a three-stage coding schema and trains supervised classifiers to extract self-reported experiences, attitudes, and themes from over 10,000 comments. The results show that about 19–20% report personal use with generally positive attitudes, but privacy concerns, lack of professional oversight, and questions about therapeutic depth remain prominent, with no clear evidence of alignment to a formal therapeutic framework. The study highlights the relevance of AI in everyday mental health discourse while underscoring the need for clinical and ethical scrutiny and more rigorous, non-social-media research to assess effectiveness and safety.

Abstract

The emergence of generative AI chatbots such as ChatGPT has prompted growing public and academic interest in their role as informal mental health support tools. While early rule-based systems have been around for several years, large language models (LLMs) offer new capabilities in conversational fluency, empathy simulation, and availability. This study explores how users engage with LLMs as mental health tools by analyzing over 10,000 TikTok comments from videos referencing LLMs as mental health tools. Using a self-developed tiered coding schema and supervised classification models, we identify user experiences, attitudes, and recurring themes. Results show that nearly 20% of comments reflect personal use, with these users expressing overwhelmingly positive attitudes. Commonly cited benefits include accessibility, emotional support, and perceived therapeutic value. However, concerns around privacy, generic responses, and the lack of professional oversight remain prominent. It is important to note that the user feedback does not indicate which therapeutic framework, if any, the LLM-generated output aligns with. While the findings underscore the growing relevance of AI in everyday practices, they also highlight the urgent need for clinical and ethical scrutiny in the use of AI for mental health support.

"It Listens Better Than My Therapist": Exploring Social Media Discourse on LLMs as Mental Health Tool

TL;DR

This paper investigates how people discuss using large language models (LLMs) for mental health support on TikTok. It develops a three-stage coding schema and trains supervised classifiers to extract self-reported experiences, attitudes, and themes from over 10,000 comments. The results show that about 19–20% report personal use with generally positive attitudes, but privacy concerns, lack of professional oversight, and questions about therapeutic depth remain prominent, with no clear evidence of alignment to a formal therapeutic framework. The study highlights the relevance of AI in everyday mental health discourse while underscoring the need for clinical and ethical scrutiny and more rigorous, non-social-media research to assess effectiveness and safety.

Abstract

The emergence of generative AI chatbots such as ChatGPT has prompted growing public and academic interest in their role as informal mental health support tools. While early rule-based systems have been around for several years, large language models (LLMs) offer new capabilities in conversational fluency, empathy simulation, and availability. This study explores how users engage with LLMs as mental health tools by analyzing over 10,000 TikTok comments from videos referencing LLMs as mental health tools. Using a self-developed tiered coding schema and supervised classification models, we identify user experiences, attitudes, and recurring themes. Results show that nearly 20% of comments reflect personal use, with these users expressing overwhelmingly positive attitudes. Commonly cited benefits include accessibility, emotional support, and perceived therapeutic value. However, concerns around privacy, generic responses, and the lack of professional oversight remain prominent. It is important to note that the user feedback does not indicate which therapeutic framework, if any, the LLM-generated output aligns with. While the findings underscore the growing relevance of AI in everyday practices, they also highlight the urgent need for clinical and ethical scrutiny in the use of AI for mental health support.

Paper Structure

This paper contains 21 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. LLMs as a Mental Health Tool Discussed by Mental Health Professionals
  3. LLMs as a Mental Health Tool Discussed by Users and Patients
  4. Data and Methods Including Previous Research with TikTok Data
  5. TikTok as a Platform
  6. Research Utilizing TikTok Data
  7. Sampling and Sample Characteristics
  8. Developing a Coding Schema to Capture User Perspectives and Experiences
  9. Tiered Classification Schema
  10. Stage 1: Own Experience (Binary)
  11. Stage 2: Opinion on Using ChatGPT as a Therapist
  12. Stage 3: Characteristics of User Perspectives
  13. Results
  14. Usage of AI as a mental health tool and general attitudes
  15. Recurring themes and topics in the comments
  16. ...and 6 more sections

Figures (2)

  • Figure 1: Classification pipeline
  • Figure 2: Sankey diagram of experiences, attitudes and topics mentioned. The comments that were not directly related to LLMs as mental health tool (code:-99) were filtered out here.