Triplet-Structured Knowledge Integration for Multi-Turn Medical Reasoning

TL;DR

TriMediQ tackles the challenge of reliable multi-turn clinical reasoning by converting dialogue into a triplet-based knowledge graph and using a trainable projection module to inject structured, graph-aware information into a frozen expert LLM. The two-stage approach first trains a graph-encoder + projector on a triplet KG and then applies this projection during inference to guide multi-hop reasoning without altering LLM weights. Across iMedQA and iCRAFT-MD benchmarks and multiple LLM backbones, TriMediQ yields up to $10.4\%$ absolute accuracy gains over baselines, while demonstrating clear benefits from explicit relational modeling vs. simple prompt-based integration. The findings highlight the practical potential of structured knowledge representations to enhance the reliability and depth of AI-assisted multi-turn medical dialogue systems.

Abstract

Large Language Models (LLMs) have shown strong performance on static medical Question Answering (QA) tasks, yet their reasoning often deteriorates in multi-turn clinical dialogues where patient information is scattered across turns. This paper introduces TriMediQ, a triplet-structured approach that enhances the reasoning reliability of LLMs through explicit knowledge integration. TriMediQ first employs a frozen triplet extraction LLM to convert patient responses into clinically grounded triplets, ensuring factual precision via constrained prompting. These triplets are incorporated into a patient-specific Knowledge Graph (KG), from which a trainable projection module consisting of a graph encoder and a projector captures relational dependencies while keeping all LLM parameters frozen. During inference, the projection module guides multi-hop reasoning over the KG, enabling coherent clinical dialogue understanding. Experiments on two interactive medical QA benchmarks show that TriMediQ achieves up to 10.4\% improvement in accuracy over five existing baselines on the iMedQA dataset. These results demonstrate that structuring patient information as triplets can effectively improve the reasoning capability of LLMs in multi-turn medical QA.

Figures (5)

• Figure 1: Interactive Medical Question Answering: a. The MEDIQ framework requires the expert to identify key clinical facts (e.g., fatigue, night sweats, rifampin) directly from dialogue. b. The proposed TriMediQ highlights these clinical facts as triplet-based KG, where pink nodes denote different clinical facts in the KG.
• Figure 2: Overview of the TriMediQ for Interactive Medical QA: The framework consists of two stages: (i) fine-tuning a projection module with a structured triplet graph; (ii) using the trained projection to support inference during interactive QA. The Triplet Generator converts patient responses into UMLS-style triplets, while the Projection Module (graph encoder + projector) encodes these triplets and injects them into the frozen expert LLM via prefix tuning.
• Figure 3: Accuracy (%) comparison between IP and PGT across multiple LLMs on iCRAFT-MD (left) and iMEDQA (right).
• Figure 4: Accuracy (%) comparison between PT and PGT across multiple LLMs on iCRAFT-MD (left) and iMEDQA (right).
• Figure 5: Accuracy (%) as a function of dialogue turns. TriMediQ outperforms MEDIQ with consistent gains as the number of turns increases.