Linking Knowledge to Care: Knowledge Graph-Augmented Medical Follow-Up Question Generation

TL;DR

A Knowledge Graph-augmented LLM with active in-context learning to generate relevant and important follow-up questions, KG-Followup, serving as a critical module for the pre-diagnostic assessment.

Abstract

Clinical diagnosis is time-consuming, requiring intensive interactions between patients and medical professionals. While large language models (LLMs) could ease the pre-diagnostic workload, their limited domain knowledge hinders effective medical question generation. We introduce a Knowledge Graph-augmented LLM with active in-context learning to generate relevant and important follow-up questions, KG-Followup, serving as a critical module for the pre-diagnostic assessment. The structured medical domain knowledge graph serves as a seamless patch-up to provide professional domain expertise upon which the LLM can reason. Experiments demonstrate that KG-Followup outperforms state-of-the-art methods by 5% - 8% on relevant benchmarks in recall.

Figures (6)

• Figure 1: Our method generates preliminary questions from a patient’s message and EHR, extracts medical entities via an LLM, applies (i) EHR-guided associative concept retrieval, (ii) DDX-guided reasoning path search, and (iii) KG-informed active in-context learning, and finally filters the results into a controlled set of follow-up questions.
• Figure 2: Ablation study of different generation signals, evaluated using Claude Haiku on the FollowupBench.
• Figure 2: Analysis of ICL and Zero-Shot-$k$ trends with Claude Sonnet on ClinicalInquiryBench: Figure (a) shows the performance across shot numbers; Figure (b) shows the effect of controlled Zero-Shot-$k$.
• Figure 3: Case Study.Red and Green indicate questions generated from KG-retrieved symptom signals and the LLM’s internal knowledge, respectively.
• Figure 4: A curated instance illustration.