Beyond Direct Diagnosis: LLM-based Multi-Specialist Agent Consultation for Automatic Diagnosis
Haochun Wang, Sendong Zhao, Zewen Qiang, Nuwa Xi, Bing Qin, Ting Liu
TL;DR
The paper addresses automatic diagnosis by modeling real-world clinical workflows as multi-specialist consultations. It introduces the Agent-derived Multi-Specialist Consultation (AMSC) framework, which uses tuning-free LLMs as a general practitioner and disease-specific specialist agents, framing diagnosis as a hierarchical MCQA and fusing predictions with a self-attention based Adaptive Probability Distribution Fusion (APDF) mechanism; final predictions are computed as $p_{disease_i} = \frac{\texttt{LLM}(q,\text{opts}; i)}{\sum_{j} \texttt{LLM}(q,\text{opts}; j)}$ and refined through $Q,K,V$ projections to produce the final $p_{disease}$. The approach achieves up to a 3.2% accuracy boost over strong baselines across three real-world datasets, while dramatically reducing training time (up to 95.5%) and trainable parameters (up to 99.99%), and provides insight into the role of implicit symptoms and knowledge alignment. By leveraging open-source LLMs deployed privately, AMSC offers a privacy-conscious, scalable path toward practical AI-assisted diagnosis, with explicit symptoms shown to carry substantial diagnostic signal and implicit symptoms offering limited additional benefit in some configurations.
Abstract
Automatic diagnosis is a significant application of AI in healthcare, where diagnoses are generated based on the symptom description of patients. Previous works have approached this task directly by modeling the relationship between the normalized symptoms and all possible diseases. However, in the clinical diagnostic process, patients are initially consulted by a general practitioner and, if necessary, referred to specialists in specific domains for a more comprehensive evaluation. The final diagnosis often emerges from a collaborative consultation among medical specialist groups. Recently, large language models have shown impressive capabilities in natural language understanding. In this study, we adopt tuning-free LLM-based agents as medical practitioners and propose the Agent-derived Multi-Specialist Consultation (AMSC) framework to model the diagnosis process in the real world by adaptively fusing probability distributions of agents over potential diseases. Experimental results demonstrate the superiority of our approach compared with baselines. Notably, our approach requires significantly less parameter updating and training time, enhancing efficiency and practical utility. Furthermore, we delve into a novel perspective on the role of implicit symptoms within the context of automatic diagnosis.