Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Advancing Mental Health Pre-Screening: A New Custom GPT for Psychological Distress Assessment

Jinwen Tang, Yi Shang

TL;DR

The paper addresses scalable, private pre-screening of mental health conditions by introducing Psycho Analyst, a GPT-4–based system integrated with DSM-5 criteria and PHQ-8 scoring. It employs a dual-task framework to perform binary classification and a three-stage PHQ-8 computation on DAIC-WOZ transcripts, validated against multiple knowledge configurations and baselines. The study reports state-of-the-art performance on DAIC-WOZ, notably when DSM-5 and PHQ-8, plus detailed data descriptions and expanded training, are combined, achieving high F1 and Macro-F1 scores and low PHQ-8 error metrics. The work suggests significant practical impact for at-home screening and clinician support, while acknowledging limitations such as output inconsistencies and ethical considerations, and outlining future directions like social-media analysis and an interactive interview chatbot.

Abstract

This study introduces 'Psycho Analyst', a custom GPT model based on OpenAI's GPT-4, optimized for pre-screening mental health disorders. Enhanced with DSM-5, PHQ-8, detailed data descriptions, and extensive training data, the model adeptly decodes nuanced linguistic indicators of mental health disorders. It utilizes a dual-task framework that includes binary classification and a three-stage PHQ-8 score computation involving initial assessment, detailed breakdown, and independent assessment, showcasing refined analytic capabilities. Validation with the DAIC-WOZ dataset reveals F1 and Macro-F1 scores of 0.929 and 0.949, respectively, along with the lowest MAE and RMSE of 2.89 and 3.69 in PHQ-8 scoring. These results highlight the model's precision and transformative potential in enhancing public mental health support, improving accessibility, cost-effectiveness, and serving as a second opinion for professionals.

Advancing Mental Health Pre-Screening: A New Custom GPT for Psychological Distress Assessment

TL;DR

The paper addresses scalable, private pre-screening of mental health conditions by introducing Psycho Analyst, a GPT-4–based system integrated with DSM-5 criteria and PHQ-8 scoring. It employs a dual-task framework to perform binary classification and a three-stage PHQ-8 computation on DAIC-WOZ transcripts, validated against multiple knowledge configurations and baselines. The study reports state-of-the-art performance on DAIC-WOZ, notably when DSM-5 and PHQ-8, plus detailed data descriptions and expanded training, are combined, achieving high F1 and Macro-F1 scores and low PHQ-8 error metrics. The work suggests significant practical impact for at-home screening and clinician support, while acknowledging limitations such as output inconsistencies and ethical considerations, and outlining future directions like social-media analysis and an interactive interview chatbot.

Abstract

This study introduces 'Psycho Analyst', a custom GPT model based on OpenAI's GPT-4, optimized for pre-screening mental health disorders. Enhanced with DSM-5, PHQ-8, detailed data descriptions, and extensive training data, the model adeptly decodes nuanced linguistic indicators of mental health disorders. It utilizes a dual-task framework that includes binary classification and a three-stage PHQ-8 score computation involving initial assessment, detailed breakdown, and independent assessment, showcasing refined analytic capabilities. Validation with the DAIC-WOZ dataset reveals F1 and Macro-F1 scores of 0.929 and 0.949, respectively, along with the lowest MAE and RMSE of 2.89 and 3.69 in PHQ-8 scoring. These results highlight the model's precision and transformative potential in enhancing public mental health support, improving accessibility, cost-effectiveness, and serving as a second opinion for professionals.
Paper Structure (16 sections, 4 figures, 9 tables)

This paper contains 16 sections, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Data Description and Preprocessing
  4. Methods
  5. Model Development
  6. Dual Task Framework
  7. Task 1: Classification
  8. Task 2: PHQ-8 Score Computation
  9. Stage 1: Initial Analysis
  10. Stage 2: Detailed Breakdown
  11. Stage 3: Independent Assessment
  12. Evaluation
  13. Results and Analysis
  14. Task 1: Classification Result
  15. Task 2: PHQ-8 Score Computation Result
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Histogram of ChatGPT token counts of combined participant dialogues in the DAIC-WOZ dataset.
  • Figure 2: Three-Stage Mental Health Evaluation Framework: Comprehensive Analysis, Re-Evaluation, and Independent Validation of Others' Opinions.
  • Figure 3: Psycho Analyst prediction accuracy on the overall DAIC-WOZ dataset for the four different background knowledge configurations and likelihood score threshold values from 3 to 7.
  • Figure 4: Frequency Distribution of Absolute Differences Across Three Stages of Evaluation for Various Models on the DAIC-WOZ Test Set.