A Two-Stage Proactive Dialogue Generator for Efficient Clinical Information Collection Using Large Language Model

TL;DR

This work proposes a diagnostic dialogue system to automate the patient information collection procedure, and shows that the model can generate clinical queries that mimic the conversation style of real doctors, with efficient fluency, professionalism, and safety, while effectively collecting relevant disease diagnostic information.

Abstract

Efficient patient-doctor interaction is among the key factors for a successful disease diagnosis. During the conversation, the doctor could query complementary diagnostic information, such as the patient's symptoms, previous surgery, and other related information that goes beyond medical evidence data (test results) to enhance disease diagnosis. However, this procedure is usually time-consuming and less-efficient, which can be potentially optimized through computer-assisted systems. As such, we propose a diagnostic dialogue system to automate the patient information collection procedure. By exploiting medical history and conversation logic, our conversation agents, particularly the doctor agent, can pose multi-round clinical queries to effectively collect the most relevant disease diagnostic information. Moreover, benefiting from our two-stage recommendation structure, carefully designed ranking criteria, and interactive patient agent, our model is able to overcome the under-exploration and non-flexible challenges in dialogue generation. Our experimental results on a real-world medical conversation dataset show that our model can generate clinical queries that mimic the conversation style of real doctors, with efficient fluency, professionalism, and safety, while effectively collecting relevant disease diagnostic information.

Figures (4)

• Figure 1: A comparison between existing a) Question-Answering (QA) System and b) Proactive Dialogue System. The traditional question-answering system answers the questions from patients in a passive way. Our proposed Proactive Dialogue System could proactively generate queries, leading to a dialogue that collects diagnostic information that supports Diagnosis and Treatment.
• Figure 2: The algorithm diagram of the proposed proactive dialogue system framework. (a): Structure of the proposed proactive dialogue generator. (b): Fine-tuning stage of doctor agent. The doctor agent has access to the patient's query and medical history of the patient. The proactive dialogue generator produces responses from patients. In implementation, this dialogue generation process is optimized by the process of fine-tuning the doctor agent.
• Figure 3: The prompt used in this paper. (a): The prompts used to calculate the consistency between the generated dialogue and the medical history. (b): The prompt used to calculate the quality of the generated dialogue by the nursing agent. (c): The prompt for calculating high-level metrics, including Fluency, Professionalism, and Safety, of the generated dialogue. (d): The prompt to extract diagnostic information from the generated dialogue. We use the first data entry in the testing set for the example in the prompt. (e): The prompt for the patient agent. The patient agent has access to the medical history and generates responses which can be follow-up questions regarding their medical conditions or an answer to the doctor agent's previous question.
• Figure 4: Representative examples of patient-doctors dialogues. For a query from a patient, two variations of our model (finetuning+candidate ranking and finetuning) generate responses. The reference response is shown in also demonstrated. The proactive questions are highlighted in red color. The demonstrations use the first round of dialogue from the real-world conversation dataset. In these cases, our methods (finetuning+candidate ranking) and (finetuning+candidate ranking+patient agent) generate the same response.