TRUST: An LLM-Based Dialogue System for Trauma Understanding and Structured Assessments

TL;DR

TRUST introduces an LLM-based dialogue system designed to conduct formal diagnostic interviews and structured PTSD assessments using CAPS-DSM-5 criteria. The framework deploys cooperative LLM modules, a dedicated Dialogue Acts schema, and a novel patient-simulation approach grounded in real interview transcripts to enable robust evaluation. Expert assessments show TRUST performing comparably to clinicians, while automated evaluations reveal limitations and the need for human oversight in clinical contexts. The work lays a foundation for scalable, accessible, structured diagnostic interviews across PTSD and potentially other mental health conditions, with careful attention to ethics and data privacy.

Abstract

Objectives: While Large Language Models (LLMs) have been widely used to assist clinicians and support patients, no existing work has explored dialogue systems for standard diagnostic interviews and assessments. This study aims to bridge the gap in mental healthcare accessibility by developing an LLM-powered dialogue system that replicates clinician behavior. Materials and Methods: We introduce TRUST, a framework of cooperative LLM modules capable of conducting formal diagnostic interviews and assessments for Post-Traumatic Stress Disorder (PTSD). To guide the generation of appropriate clinical responses, we propose a Dialogue Acts schema specifically designed for clinical interviews. Additionally, we develop a patient simulation approach based on real-life interview transcripts to replace time-consuming and costly manual testing by clinicians. Results: A comprehensive set of evaluation metrics is designed to assess the dialogue system from both the agent and patient simulation perspectives. Expert evaluations by conversation and clinical specialists show that TRUST performs comparably to real-life clinical interviews. Discussion: Our system performs at the level of average clinicians, with room for future enhancements in communication styles and response appropriateness. Conclusions: Our TRUST framework shows its potential to facilitate mental healthcare availability.

Figures (5)

• Figure 1: Overview of the TRUST system in an example variable session. Database stores metadata for each variable (e.g., symptoms to be assessed). Conversation and Assessment are LLM-powered cooperative AI modules that control dialogue flow, generate clinician responses, and perform assessments. These modules use variable metadata and follow the numbered steps ①-⑤ to generate the conversation, with grey steps indicating optional actions. Simulate Patient is used only during system evaluation, and Assess Variable is triggered once sufficient information has been collected. Variable Session illustrates a generated conversation for the example variable, which assesses the intensity of the patient's avoidance of trauma-related thoughts or feelings in the past month. Variable Assessment presents the corresponding assessment result predicted by the system.
• Figure 2: Dialogue flowchart design for each variable. This architecture integrates Modules for Database interaction, conversation management and assessment logic to ensure adaptive and contextually appropriate interactions tailored for individuals. Decision node represents key decision points, while Process node indicate processing functions that manage dialogue generation and assessment tasks.
• Figure 3: An example of patient simulation in TRUST. The left side shows segments from a real-life PTSD interview transcript, while the right side presents the generated dialogue produced by TRUST. Both the clinician and patient responses on the right side are generated by our system.
• Figure 4: Evalutaion results across 5 dialogues (Dialogue 1-5). The x-axis displays evaluation metrics, grouped by dialogue ID. The y-aixs shows evaluation scores, with a red dashed reference line marking the lower threshold of Equivalent Performance for the agent and Acceptable for simulation. Note that the reference line for Patient Simulation overlaps the 0.0 value line.
• Figure 5: Examples of inappropriate Communication Style and interview question. Exaggerated interpretation and unnecessary question are highlighted in orange. Next IS question is highlighted in green. Question icon indicates the PTSD symptom to assess.