Listening to Patients: A Framework of Detecting and Mitigating Patient Misreport for Medical Dialogue Generation

TL;DR

The paper tackles patient misreporting in medical dialogue systems by introducing PaMis, a framework that constructs a dialogue entity graph atop a static knowledge graph to detect misreports via graph entropy and mitigate them by generating targeted clarifying questions. Misreports are categorized into isolated, denial, and contradiction patterns, detected through one-dimensional structural entropy of the evolving entity graph, and mitigated using retrieved entity-centered knowledge from one-hop neighbors. Evaluations on MedDG and KaMed show that PaMis improves medical response quality and enables MIS-aware clarification, with notable gains when paired with GPT-4, indicating potential as a diagnostic aid that guides doctors toward more accurate assessments without replacing clinical judgment. The approach demonstrates how integrating structured entity representations with external medical knowledge can enhance the safety and effectiveness of medical dialogue systems in the presence of misreported patient information.

Abstract

Medical Dialogue Systems aim to provide automated healthcare support through patient-agent conversations. Previous efforts typically regard patients as ideal users -- one who accurately and consistently reports their health conditions. However, in reality, patients often misreport their symptoms, leading to discrepancies between their reports and actual health conditions. Overlooking patient misreport will affect the quality of healthcare consultations provided by MDS. To address this issue, we argue that MDS should ''listen to patients'' and tackle two key challenges: how to detect and mitigate patient misreport effectively. In this work, we propose PaMis, a framework of detecting and mitigating Patient Misreport for medical dialogue generation. PaMis first constructs dialogue entity graphs, then detects patient misreport based on graph entropy, and mitigates patient misreport by formulating clarifying questions. Experiments indicate that PaMis effectively enhances medical response generation, enabling models like GPT-4 to detect and mitigate patient misreports, and provide high-quality healthcare assistance.

Figures (5)

• Figure 1: Example of patient misreport in patient (P)-agent (A) conversations and a response generated by the real experienced doctor (D). When the patient with myocardial infarction misreports angina as abdominal pain, the doctor remains vigilant and asks more detailed questions to discern the patient's actual symptoms. However, the agent can easily be influenced by the patient's misreport and arbitrarily shift the focus to stomach flu.
• Figure 2: An illustration of PaMis, using the dialogue in Figure \ref{['fig:intro_example']} as an example. PaMis first constructs the entity graph, and then detects and mitigates the patient misreport based on the entity graph.
• Figure 3: Evaluation results of interactive experiment under two misreport-aware metrics: $\Delta GE$ and MR.
• Figure 4: The human evaluation results of PaMis vs. GPT-4 (w/ Gold Know.) on two datasets.
• Figure 5: Different scenarios after losing a node (using the example of 4 remaining nodes).