Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring Large Language Models for Detecting Mental Disorders

Gleb Kuzmin, Petr Strepetov, Maksim Stankevich, Natalia Chudova, Artem Shelmanov, Ivan Smirnov

TL;DR

This study tackles automatic detection of depression and anxiety from Russian texts by comparing traditional linguistic-feature ML, encoder-based transformers, and large language models across five datasets. It finds that LLMs generally outperform other approaches, especially on noisy or small datasets, though transfer from clinically diagnosed essays to questionnaire-based social media data is limited. Psycholinguistic features and encoder models can match LLM performance when trained on clinically validated depression texts, and BERT-like encoders excel on heterogeneous, noisy data. However, LLM-generated explanations lag behind clinical requirements, with substantial error types identified, indicating a need for better interpretability and clinician-in-the-loop prompting for practical use.

Abstract

This paper compares the effectiveness of traditional machine learning methods, encoder-based models, and large language models (LLMs) on the task of detecting depression and anxiety. Five Russian-language datasets were considered, each differing in format and in the method used to define the target pathology class. We tested AutoML models based on linguistic features, several variations of encoder-based Transformers such as BERT, and state-of-the-art LLMs as pathology classification models. The results demonstrated that LLMs outperform traditional methods, particularly on noisy and small datasets where training examples vary significantly in text length and genre. However, psycholinguistic features and encoder-based models can achieve performance comparable to language models when trained on texts from individuals with clinically confirmed depression, highlighting their potential effectiveness in targeted clinical applications.

Exploring Large Language Models for Detecting Mental Disorders

TL;DR

This study tackles automatic detection of depression and anxiety from Russian texts by comparing traditional linguistic-feature ML, encoder-based transformers, and large language models across five datasets. It finds that LLMs generally outperform other approaches, especially on noisy or small datasets, though transfer from clinically diagnosed essays to questionnaire-based social media data is limited. Psycholinguistic features and encoder models can match LLM performance when trained on clinically validated depression texts, and BERT-like encoders excel on heterogeneous, noisy data. However, LLM-generated explanations lag behind clinical requirements, with substantial error types identified, indicating a need for better interpretability and clinician-in-the-loop prompting for practical use.

Abstract

This paper compares the effectiveness of traditional machine learning methods, encoder-based models, and large language models (LLMs) on the task of detecting depression and anxiety. Five Russian-language datasets were considered, each differing in format and in the method used to define the target pathology class. We tested AutoML models based on linguistic features, several variations of encoder-based Transformers such as BERT, and state-of-the-art LLMs as pathology classification models. The results demonstrated that LLMs outperform traditional methods, particularly on noisy and small datasets where training examples vary significantly in text length and genre. However, psycholinguistic features and encoder-based models can achieve performance comparable to language models when trained on texts from individuals with clinically confirmed depression, highlighting their potential effectiveness in targeted clinical applications.

Paper Structure

This paper contains 32 sections, 2 figures, 21 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional and Advanced ML Methods
  4. Large Language Models
  5. Data
  6. Depression-Essays (DE)
  7. Depression-Essays (DE)
  8. Depression-Social Media (DSM)
  9. Anxiety-Letter and Anxiety Picture Description (AL and AD)
  10. Anxiety-COVID Comments (AC)
  11. Methods
  12. Linguistic Features
  13. Psycholinguistic Features
  14. Classification Setup
  15. Encoder-Based Models
  16. ...and 17 more sections

Figures (2)

  • Figure 1: The most important features for the target label generation from the test set of the DE dataset, for the predicted healthy class (left) and pathology class (right). The features are given in Russian (up) and translated into English (down).
  • Figure 2: The most important features for the explanation generation from the test set of the DE dataset, for the predicted healthy class (left) and pathology class (right). The features are given in Russian (up) and translated into English (down).