RE-MCDF: Closed-Loop Multi-Expert LLM Reasoning for Knowledge-Grounded Clinical Diagnosis

TL;DR

RE-MCDF tackles the core challenge of unreliable, hallucination-prone LLM-based clinical diagnosis from heterogeneous EMRs by introducing a closed-loop, relation-aware multi-expert framework guided by a medical knowledge graph. The primary expert generates candidate diagnoses and evidence, the laboratory expert reweights heterogeneous indicators, and a multi-relation group enforces inter-disease constraints to ensure logical consistency. Extensive experiments on NEEMRs and XMEMRs show state-of-the-art performance and robust generalization, with ablation studies highlighting the critical roles of dynamic evidence weighting and explicit logical constraints. The framework promises more trustworthy, explainable, and clinically aligned diagnostic reasoning suitable for neurology and potentially beyond.

Abstract

Electronic medical records (EMRs), particularly in neurology, are inherently heterogeneous, sparse, and noisy, which poses significant challenges for large language models (LLMs) in clinical diagnosis. In such settings, single-agent systems are vulnerable to self-reinforcing errors, as their predictions lack independent validation and can drift toward spurious conclusions. Although recent multi-agent frameworks attempt to mitigate this issue through collaborative reasoning, their interactions are often shallow and loosely structured, failing to reflect the rigorous, evidence-driven processes used by clinical experts. More fundamentally, existing approaches largely ignore the rich logical dependencies among diseases, such as mutual exclusivity, pathological compatibility, and diagnostic confusion. This limitation prevents them from ruling out clinically implausible hypotheses, even when sufficient evidence is available. To overcome these, we propose RE-MCDF, a relation-enhanced multi-expert clinical diagnosis framework. RE-MCDF introduces a generation--verification--revision closed-loop architecture that integrates three complementary components: (i) a primary expert that generates candidate diagnoses and supporting evidence, (ii) a laboratory expert that dynamically prioritizes heterogeneous clinical indicators, and (iii) a multi-relation awareness and evaluation expert group that explicitly enforces inter-disease logical constraints. Guided by a medical knowledge graph (MKG), the first two experts adaptively reweight EMR evidence, while the expert group validates and corrects candidate diagnoses to ensure logical consistency. Extensive experiments on the neurology subset of CMEMR (NEEMRs) and on our curated dataset (XMEMRs) demonstrate that RE-MCDF consistently outperforms state-of-the-art baselines in complex diagnostic scenarios.

Figures (3)

• Figure 1: Architecture of RE-MCDF. Consists of a primary expert that generates initial diagnosis–evidence pairs from EMRs, and a laboratory expert that dynamically weights heterogeneous clinical indicators (e.g., abnormal laboratory findings). A MKG is then used to expand candidate coverage and provide structured relational paths to four collaborative experts: single-disease assessment, disease logic integration, confusion detection, and logical adjustment. Together, these experts enforce logical consistency through relation-aware scoring and trigger closed-loop feedback when conflicts are detected (e.g., mutually exclusive diseases with similar confidence), enabling traceable and clinically plausible diagnostic reasoning.
• Figure 2: Performance analysis and human evaluation. Left subplot: Impact of MKG supplement depth $\hat{k}_\text{sup}$ on retrieval and verification ($\hat{k}_\text{sup}$ = 1 for (i), $\hat{k}_\text{sup}$ = 2 for (ii)). Right subplot: Manual evaluation of reasoning trajectories, where $\mathcal{A}_{\text{conf}}$ and $\mathcal{A}_{\text{exc}}$ are measured by physician approval rate, and $\mathcal{A}_{\text{adj}}$ by accuracy rate (Based on Qwen2.5).
• Figure 3: Illustrative case study of RE-MCDF: Integrating clinical evidence with relation-aware reasoning.