Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Honesty-Aware Multi-Agent Framework for High-Fidelity Synthetic Data Generation in Digital Psychiatric Intake Doctor-Patient Interactions

Xinyuan Zhang, Zijian Wang, Chang Dao, Juexiao Zhou

TL;DR

This work tackles data scarcity and dishonest reporting in psychiatric intake by introducing MentalAED, an honesty-aware four-role multi-agent framework that synthesizes high-fidelity intake traces from DAIC-WOZ. The system links patient profiles, self-rated scales, rater-administered scales, semi-structured interviews, and diagnostic summaries under topic-dependent honesty states (concealment or exaggeration). It features four specialized agents (Assessor, Patient, Evaluator, Diagnostician) and a CoT-enabled Evaluator that improves diagnostic fidelity and rapport-building, validated through diagnostic alignment, ablation studies, human expert evaluation, and LLM-based baselines. The resulting synthetic corpus enables controlled investigations of dishonesty-aware assessment and provides a training/evaluation resource for adaptive clinical dialogue agents, with implications for scalable and privacy-preserving mental health tools.

Abstract

Data scarcity and unreliable self-reporting -- such as concealment or exaggeration -- pose fundamental challenges to psychiatric intake and assessment. We propose a multi-agent synthesis framework that explicitly models patient deception to generate high-fidelity, publicly releasable synthetic psychiatric intake records. Starting from DAIC-WOZ interviews, we construct enriched patient profiles and simulate a four-role workflow: a \emph{Patient} completes self-rated scales and participates in a semi-structured interview under a topic-dependent honesty state; an \emph{Assessor} selects instruments based on demographics and chief complaints; an \emph{Evaluator} conducts the interview grounded in rater-administered scales, tracks suspicion, and completes ratings; and a \emph{Diagnostician} integrates all evidence into a diagnostic summary. Each case links the patient profile, self-rated and rater-administered responses, interview transcript, diagnostic summary, and honesty state. We validate the framework through four complementary evaluations: diagnostic consistency and severity grading, chain-of-thought ablations, human evaluation of clinical realism and dishonesty modeling, and LLM-based comparative evaluation. The resulting corpus spans multiple disorders and severity levels, enabling controlled study of dishonesty-aware psychiatric assessment and the training and evaluation of adaptive dialogue agents.

Honesty-Aware Multi-Agent Framework for High-Fidelity Synthetic Data Generation in Digital Psychiatric Intake Doctor-Patient Interactions

TL;DR

This work tackles data scarcity and dishonest reporting in psychiatric intake by introducing MentalAED, an honesty-aware four-role multi-agent framework that synthesizes high-fidelity intake traces from DAIC-WOZ. The system links patient profiles, self-rated scales, rater-administered scales, semi-structured interviews, and diagnostic summaries under topic-dependent honesty states (concealment or exaggeration). It features four specialized agents (Assessor, Patient, Evaluator, Diagnostician) and a CoT-enabled Evaluator that improves diagnostic fidelity and rapport-building, validated through diagnostic alignment, ablation studies, human expert evaluation, and LLM-based baselines. The resulting synthetic corpus enables controlled investigations of dishonesty-aware assessment and provides a training/evaluation resource for adaptive clinical dialogue agents, with implications for scalable and privacy-preserving mental health tools.

Abstract

Data scarcity and unreliable self-reporting -- such as concealment or exaggeration -- pose fundamental challenges to psychiatric intake and assessment. We propose a multi-agent synthesis framework that explicitly models patient deception to generate high-fidelity, publicly releasable synthetic psychiatric intake records. Starting from DAIC-WOZ interviews, we construct enriched patient profiles and simulate a four-role workflow: a \emph{Patient} completes self-rated scales and participates in a semi-structured interview under a topic-dependent honesty state; an \emph{Assessor} selects instruments based on demographics and chief complaints; an \emph{Evaluator} conducts the interview grounded in rater-administered scales, tracks suspicion, and completes ratings; and a \emph{Diagnostician} integrates all evidence into a diagnostic summary. Each case links the patient profile, self-rated and rater-administered responses, interview transcript, diagnostic summary, and honesty state. We validate the framework through four complementary evaluations: diagnostic consistency and severity grading, chain-of-thought ablations, human evaluation of clinical realism and dishonesty modeling, and LLM-based comparative evaluation. The resulting corpus spans multiple disorders and severity levels, enabling controlled study of dishonesty-aware psychiatric assessment and the training and evaluation of adaptive dialogue agents.
Paper Structure (76 sections, 2 equations, 4 figures, 7 tables)

This paper contains 76 sections, 2 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM-Based Psychiatric Applications and Datasets
  4. Digital Patients and Response Styles
  5. Synthetic Patient–Scale Dataset Construction
  6. Data Resource and Preprocessing
  7. MentalAED
  8. Framework Overview
  9. Agent Design
  10. Assessor Agent
  11. Patient Agent.
  12. Evaluator Agent.
  13. Diagnostician Agent
  14. The Simulated Corpus
  15. Experiments and Evaluation
  16. ...and 61 more sections

Figures (4)

  • Figure 1: Architecture of the proposed multi-agent simulation framework--MentalAED.
  • Figure 2: Overview of the Doctor-Patient Dialogue Simulation. The left panel illustrates the agents' interaction mechanism; the right panel presents a representative excerpt of the synthesized dialogue.
  • Figure 3: Comparative Trust Dynamics over Interaction Turns.
  • Figure 4: Distribution of Net Trust Gain ($\Delta \text{Trust}$). Comparison of the total trust accumulated from the initial state to the end of the session.