AI-Enhanced Cognitive Behavioral Therapy: Deep Learning and Large Language Models for Extracting Cognitive Pathways from Social Media Texts

TL;DR

The paper tackles the problem of extracting cognitive pathways from social-media text to support Cognitive Behavioral Therapy by modeling four ABCD-based parent categories with nineteen subcategories. It frames two tasks—hierarchical text classification (HTC) and abstractive text summarization—and compares deep learning (ERNIE 3.0) with large language models (GPT-4/GPT-3.5) using data from Weibo and Reddit, with ERNIE 3.0 performing best on HTC and GPT-4 excelling at summarization. Key findings include an overall HTC micro-F1 of 62.34% for DL, and GPT-4 achieving Rouge-1 54.92 and Rouge-2 30.86 on summarization, though LLMs may hallucinate; both model families are open-sourced. The work demonstrates a practical pathway to accelerate CBT interventions via AI-assisted cognitive-pathway extraction, while highlighting trade-offs between accuracy and hallucination risk and providing a foundation for future personalized, online psychotherapy tools.

Abstract

Cognitive Behavioral Therapy (CBT) is an effective technique for addressing the irrational thoughts stemming from mental illnesses, but it necessitates precise identification of cognitive pathways to be successfully implemented in patient care. In current society, individuals frequently express negative emotions on social media on specific topics, often exhibiting cognitive distortions, including suicidal behaviors in extreme cases. Yet, there is a notable absence of methodologies for analyzing cognitive pathways that could aid psychotherapists in conducting effective interventions online. In this study, we gathered data from social media and established the task of extracting cognitive pathways, annotating the data based on a cognitive theoretical framework. We initially categorized the task of extracting cognitive pathways as a hierarchical text classification with four main categories and nineteen subcategories. Following this, we structured a text summarization task to help psychotherapists quickly grasp the essential information. Our experiments evaluate the performance of deep learning and large language models (LLMs) on these tasks. The results demonstrate that our deep learning method achieved a micro-F1 score of 62.34% in the hierarchical text classification task. Meanwhile, in the text summarization task, GPT-4 attained a Rouge-1 score of 54.92 and a Rouge-2 score of 30.86, surpassing the experimental deep learning model's performance. However, it may suffer from an issue of hallucination. We have made all models and codes publicly available to support further research in this field.

Figures (5)

• Figure 1: This figure outlines the workflow of the research. Initially, experts at various experience levels collaborate to annotate the data. Subsequently, different methods (deep learning model or LLMs) are developed to classify the cognitive categories from the user statement. In the final step, the results of these pathways are summarized to produce a final cognitive pathway to help psychotherapists understand quickly.
• Figure 2: The figure presents an English translation of a user's statement from Chinese social media, highlighting suicidal ideation and cognitive distortions. It demonstrates how the user's statement is segmented into ABCD components in CBT. By analyzing statements, psychotherapist can effectively counter irrational belief (B) and prevent habitual disputation (D) through effective disputation (D), thereby gradually amending cognitive distortions. Note that due to space constraints, some irrelevant information in the user statement is not shown in the translation.
• Figure 3: Distribution of sentence lengths in the dataset for original sentence lengths (a) and human-written reference summaries (b).
• Figure 4: Examples of errors in the hierarchical text classification task using the ERNIE 3.0 model are presented. For a clean presentation, we have only presented the portion relevant to the error and have not depicted the complete cognitive pathway extraction results.
• Figure 5: Output of different models in the text summarization task.