Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM Questionnaire Completion for Automatic Psychiatric Assessment

Gony Rosenman, Lior Wolf, Talma Hendler

TL;DR

The paper tackles the challenge of converting unstructured psychiatric interviews into standardized measures by introducing LMIQ, an impersonation-based questionnaire completion pipeline that converts transcripts into a $135$-dimensional feature vector of LLM-generated responses. This feature vector feeds a Random Forest regressor to predict $PHQ-8$ and $PCL-C$ scores, achieving lower mean-squared errors than multiple baselines on the $E$-DAIC (and related) datasets, with performance enhanced by using the Mixtral 7Bx8 backbone. The work demonstrates that impersonation-based questionnaire completion can bridge narrative interviews and data-driven psychiatric assessment, offering a potentially scalable framework while acknowledging ethical, privacy, and generalizability considerations. Overall, LMIQ provides a novel direction for leveraging LLMs to extract clinically meaningful signals from interview text, potentially augmenting diagnostic workflows with data-driven insights under careful safeguards.

Abstract

We employ a Large Language Model (LLM) to convert unstructured psychological interviews into structured questionnaires spanning various psychiatric and personality domains. The LLM is prompted to answer these questionnaires by impersonating the interviewee. The obtained answers are coded as features, which are used to predict standardized psychiatric measures of depression (PHQ-8) and PTSD (PCL-C), using a Random Forest regressor. Our approach is shown to enhance diagnostic accuracy compared to multiple baselines. It thus establishes a novel framework for interpreting unstructured psychological interviews, bridging the gap between narrative-driven and data-driven approaches for mental health assessment.

LLM Questionnaire Completion for Automatic Psychiatric Assessment

TL;DR

The paper tackles the challenge of converting unstructured psychiatric interviews into standardized measures by introducing LMIQ, an impersonation-based questionnaire completion pipeline that converts transcripts into a -dimensional feature vector of LLM-generated responses. This feature vector feeds a Random Forest regressor to predict and scores, achieving lower mean-squared errors than multiple baselines on the -DAIC (and related) datasets, with performance enhanced by using the Mixtral 7Bx8 backbone. The work demonstrates that impersonation-based questionnaire completion can bridge narrative interviews and data-driven psychiatric assessment, offering a potentially scalable framework while acknowledging ethical, privacy, and generalizability considerations. Overall, LMIQ provides a novel direction for leveraging LLMs to extract clinically meaningful signals from interview text, potentially augmenting diagnostic workflows with data-driven insights under careful safeguards.

Abstract

We employ a Large Language Model (LLM) to convert unstructured psychological interviews into structured questionnaires spanning various psychiatric and personality domains. The LLM is prompted to answer these questionnaires by impersonating the interviewee. The obtained answers are coded as features, which are used to predict standardized psychiatric measures of depression (PHQ-8) and PTSD (PCL-C), using a Random Forest regressor. Our approach is shown to enhance diagnostic accuracy compared to multiple baselines. It thus establishes a novel framework for interpreting unstructured psychological interviews, bridging the gap between narrative-driven and data-driven approaches for mental health assessment.
Paper Structure (20 sections, 1 figure, 6 tables)

This paper contains 20 sections, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Experiments
  5. Results and Discussion
  6. Ablating the questionnaires
  7. Conclusions
  8. Limitations
  9. Ethics and Impact Statement
  10. Acknowledgements
  11. The full prompts used
  12. System prompt - Question Impersonation
  13. System Prompt - Analysis
  14. Sample output by MentaLLama
  15. The full list of the questionnaires used.
  16. ...and 5 more sections

Figures (1)

  • Figure 1: LMIQ Pipeline Overview. A. Top. Preparation of the Extended-Daic dataset including 275 subjects with psychological interview transcripts, along with PTSD and Depression scores derived from PCL-C and PHQ-8 assessments. Bottom. Instruction for Chat-GPT 4 to develop five-item questionnaires spanning multiple mental health and personality domains. B. Main Prompt Logic. Merging of a task description with a psychological interview transcript and a questionnaire to generate five impersonated responses. Aggregate across all subjects and questionnaires. C. Training of a Random Forest model using the questionnaire responses ($d=135$) to accurately predict the original assessment scores.