Depression Diagnosis Dialogue Simulation: Self-improving Psychiatrist with Tertiary Memory

TL;DR

The Agent Mental Clinic is introduced, a self-improving conversational agent system designed to enhance depression diagnosis through simulated dialogues between patient and psychiatrist agents that can be a promising optimization method for aligning LLMs with real-life distribution in specific domains without modifying the weights of LLMs.

Abstract

Mental health issues, particularly depressive disorders, present significant challenges in contemporary society, necessitating the development of effective automated diagnostic methods. This paper introduces the Agent Mental Clinic (AMC), a self-improving conversational agent system designed to enhance depression diagnosis through simulated dialogues between patient and psychiatrist agents. To enhance the dialogue quality and diagnosis accuracy, we design a psychiatrist agent consisting of a tertiary memory structure, a dialogue control and reflect plugin that acts as ``supervisor'' and a memory sampling module, fully leveraging the skills reflected by the psychiatrist agent, achieving great accuracy on depression risk and suicide risk diagnosis via conversation. Experiment results on datasets collected in real-life scenarios demonstrate that the system, simulating the procedure of training psychiatrists, can be a promising optimization method for aligning LLMs with real-life distribution in specific domains without modifying the weights of LLMs, even when only a few representative labeled cases are available.

Figures (7)

• Figure 1: Agent Mental Clinic (AMC). The symptom list is collected during the conversation between the psychiatrist agent and the patient agent, with the guidance of the supervisor. Dialog History, Electronic Medical Records (EMR), and Skills are hierarchical memory layers of the psychiatrist agent. These layers are progressively refined throughout the depression diagnosis session.
• Figure 2: The overview of AMC. Agents are divided into patient agents and psychiatrist agents, while the supervisor plugin is attached to the psychiatrist agents to control the dialogue process. D4 database is a dataset including patient portraits, doctor-patient dialogue, and patients' diagnostic summaries.
• Figure 3: Patient Profile Examples. We select two patient agents to illustrate. The profiles of patient agents are generated from cases of D$^4$.
• Figure 4: The Tertiary Memory Structure of AMC. The utterance of the diagnosis conversation will be stored in Dialogue History. The whole dialogue history in the session will be summarized into electronic medical records (EMR). Skills are generated by the supervisor plugin. All the memory will contribute to the dialogue generation.
• Figure 5: The Results of Ablation Study on Memory Layers. The first row indicates the results based on the original dialogue history setting, while the second row implies the results based on the simulated dialogue setting. At the same time, the first column suggests the results on the train set, while the second column indicates the results of the test.