MACD: Multi-Agent Clinical Diagnosis with Self-Learned Knowledge for LLM

TL;DR

A novel Multi-Agent Clinical Diagnosis (MACD) framework, which allows LLMs to self-learn clinical knowledge via a multi-agent pipeline that summarizes, refines, and applies diagnostic insights, and presents a scalable self-learning paradigm that bridges the gap between the intrinsic knowledge of LLMs.

Abstract

Large language models (LLMs) have demonstrated notable potential in medical applications, yet they face substantial challenges in handling complex real-world clinical diagnoses using conventional prompting methods. Current prompt engineering and multi-agent approaches typically optimize isolated inferences, neglecting the accumulation of reusable clinical experience. To address this, this study proposes a novel Multi-Agent Clinical Diagnosis (MACD) framework, which allows LLMs to self-learn clinical knowledge via a multi-agent pipeline that summarizes, refines, and applies diagnostic insights. It mirrors how physicians develop expertise through experience, enabling more focused and accurate diagnosis on key disease-specific cues. We further extend it to a MACD-human collaborative workflow, where multiple LLM-based diagnostician agents engage in iterative consultations, supported by an evaluator agent and human oversight for cases where agreement is not reached. Evaluated on 4,390 real-world patient cases across seven diseases using diverse open-source LLMs (Llama-3.1 8B/70B, DeepSeek-R1-Distill-Llama 70B), MACD significantly improves primary diagnostic accuracy, outperforming established clinical guidelines with gains up to 22.3% (MACD). In direct comparison with physician-only diagnosis under the same evaluation protocol, MACD achieves comparable or superior performance, with improvements up to 16%. Furthermore, the MACD-human workflow yields an 18.6% improvement over physician-only diagnosis, demonstrating the synergistic potential of human-AI collaboration. Notably, the self-learned clinical knowledge exhibits strong cross-model stability, transferability across LLMs, and capacity for model-specific personalization.This work thus presents a scalable self-learning paradigm that bridges the gap between the intrinsic knowledge of LLMs.

Figures (14)

• Figure 1: Overview of the MACD framework. (a) The core idea of MACD is to emulate the professional development of a physician by enabling the LLM to autonomously acquire, distill, and internalize clinical knowledge from real-world diagnostic cases over time. (b) MACD is formed with a knowledge summarizer agent that identifies and extracts salient diagnostic insights from historical cases, a knowledge refiner agent that consolidates and integrates these insights into a structured, evolving knowledge memory, and a diagnostician agent that leverages this curated experience to inform and improve diagnostic reasoning. (c) A workflow comprising multiple MACD diagnostician agents based on diverse LLMs, called the MACD-human collaboration workflow. The three diagnostic agents jointly discuss the cases. The final output is provided either when agreement is reached or, if the maximum number of discussions is reached without consensus, the final diagnosis is made by human physicians.
• Figure 1: The MACD framework outperforms the zero-knowledge baseline in diagnostic accuracy. LLMs performed diagnoses based solely on their internal parameters without any knowledge or reference, to represent the model's intrinsic diagnostic capability.
• Figure 2: Comparison of diagnostic accuracy between self-learned knowledge and baseline knowledge. (a) Comparison of diagnostic accuracy across seven diseases, illustrating the consistent improvement of self-learned knowledge over baseline performance. (b) Performance assessment contrasting self-learned knowledge with Mayo Clinic's knowledge and professional knowledge, demonstrating superior diagnostic accuracy. (c) Benchmarking results highlighting the advantage of MACD over established inference methods and resource-intensive fine-tuning. (d) Comparative analysis revealing that open-source models augmented with MACD outperform the State-of-the-Art LLMs.
• Figure 2: Differences in semantic similarity of self-learned knowledge through the knowledge refiner agent. (a) Differences of the self-learned knowledge of Llama-8B model. (b) Differences of the self-learned knowledge of the DeepSeek-70B model. (c) Differences of the Self-learned knowledge of the Llama-70B model.
• Figure 3: Self-learned knowledge demonstrates a greater stability and cross-model transferability. (a) Schematic illustration of the experimental pipeline for multi-source knowledge diagnosis. (b) Performance scaling parallels zero-shot baselines, highlighting the predictability of self-learned knowledge efficacy. (c) Cross-model benchmarking demonstrates that self-learned knowledge yields performance gains across diverse LLMs, validating its robust transferability. (d) Analysis reveals a distinct model-specific preference, where models perform optimally using their internally generated knowledge. (e) RSA visualizes distinct model-specific representations while highlighting the structural consistency of diagnostic reasoning for specific diseases.