MentalGLM Series: Explainable Large Language Models for Mental Health Analysis on Chinese Social Media

TL;DR

This work addresses the need for explainable mental health analysis on Chinese social media by introducing the first Chinese interpretable mental health instruction dataset (C-IMHI) and the MentalGLM series, open-source LLMs trained to provide both predictions and explanations. The authors implement a two-stage fine-tuning pipeline, from English IMHI to Chinese C-IMHI, using LoRA on GLM-4-9b/Chat, and validate on three downstream tasks plus a clinical dataset. Across automated and human evaluations, MentalGLM achieves competitive or superior performance relative to discriminant models and other LLMs while delivering credible explanations, with strong generalization to clinical cognitive-pathway extraction. The work demonstrates practical potential for deploying explainable mental health analysis tools in real-world settings, supported by publicly available datasets and code for reproducibility and further research.

Abstract

As the prevalence of mental health challenges, social media has emerged as a key platform for individuals to express their emotions.Deep learning tends to be a promising solution for analyzing mental health on social media. However, black box models are often inflexible when switching between tasks, and their results typically lack explanations. With the rise of large language models (LLMs), their flexibility has introduced new approaches to the field. Also due to the generative nature, they can be prompted to explain decision-making processes. However, their performance on complex psychological analysis still lags behind deep learning. In this paper, we introduce the first multi-task Chinese Social Media Interpretable Mental Health Instructions (C-IMHI) dataset, consisting of 9K samples, which has been quality-controlled and manually validated. We also propose MentalGLM series models, the first open-source LLMs designed for explainable mental health analysis targeting Chinese social media, trained on a corpus of 50K instructions. The proposed models were evaluated on three downstream tasks and achieved better or comparable performance compared to deep learning models, generalized LLMs, and task fine-tuned LLMs. We validated a portion of the generated decision explanations with experts, showing promising results. We also evaluated the proposed models on a clinical dataset, where they outperformed other LLMs, indicating their potential applicability in the clinical field. Our models show strong performance, validated across tasks and perspectives. The decision explanations enhance usability and facilitate better understanding and practical application of the models. Both the constructed dataset and the models are publicly available via: https://github.com/zwzzzQAQ/MentalGLM.

Figures (5)

• Figure 1: Example of MentalGLM's prediction in the cognitive distortion classification task, including both the prediction and the explanation of its decision.
• Figure 2: The knowledge transfer process from a larger LLM (GPT-4) to a smaller one (GLM-4-9b), acting as teacher and student respectively.
• Figure 3: Evaluation results on GPT-4’s generated explanations used for C-IMHI data construction. (a) and (b) represent automated evaluation for the entire dataset, while (c) shows human evaluation on a subset of 100 samples.
• Figure 4: Expert evaluation results on 100 randomly selected prediction explanations generated by MentalGLM-chat.
• Figure S1: Prompt template used in GPT-4 to generate explanations for cognitive distortion task.