Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration

TL;DR

XMedGPT is presented, a clinician-centric, multi-modal AI assistant that integrates textual and visual interpretability to support transparent and trustworthy medical decision-making and represents a significant leap forward in clinician-centric AI integration.

Abstract

Generalist Medical AI (GMAI) systems have demonstrated expert-level performance in biomedical perception tasks, yet their clinical utility remains limited by inadequate multi-modal explainability and suboptimal prognostic capabilities. Here, we present XMedGPT, a clinician-centric, multi-modal AI assistant that integrates textual and visual interpretability to support transparent and trustworthy medical decision-making. XMedGPT not only produces accurate diagnostic and descriptive outputs, but also grounds referenced anatomical sites within medical images, bridging critical gaps in interpretability and enhancing clinician usability. To support real-world deployment, we introduce a reliability indexing mechanism that quantifies uncertainty through consistency-based assessment via interactive question-answering. We validate XMedGPT across four pillars: multi-modal interpretability, uncertainty quantification, and prognostic modeling, and rigorous benchmarking. The model achieves an IoU of 0.703 across 141 anatomical regions, and a Kendall's tau-b of 0.479, demonstrating strong alignment between visual rationales and clinical outcomes. For uncertainty estimation, it attains an AUC of 0.862 on visual question answering and 0.764 on radiology report generation. In survival and recurrence prediction for lung and glioma cancers, it surpasses prior leading models by 26.9%, and outperforms GPT-4o by 25.0%. Rigorous benchmarking across 347 datasets covers 40 imaging modalities and external validation spans 4 anatomical systems confirming exceptional generalizability, with performance gains surpassing existing GMAI by 20.7% for in-domain evaluation and 16.7% on 11,530 in-house data evaluation. Together, XMedGPT represents a significant leap forward in clinician-centric AI integration, offering trustworthy and scalable support for diverse healthcare applications.

Figures (18)

• Figure 1: a, XMedGPT is a versatile multi-modal AI framework designed to handle over 40 distinct image modalities while seamlessly integrating inputs from images, text, audio. Beyond image-level perception tasks, it excels in pixel-level analysis alongside complex prognostic modeling, including overall and progression-free survival predictions. b, The training dataset consists of 7 million image-text, including 1.6 million with pixel-level annotations. External validation spans 11 lesion types across 4 anatomical regions, including the brain, lungs, abdomen, and pelvis. c, In the human-centric evaluation, senior clinicians blindly assessed three reports from (1) human-written, (2) junior clinician + baseline AI (GMAI), and (3) junior clinician + XMedGPT. Senior clinicians provided detailed feedback to identify the best approach. d, XMedGPT-clinician collaboration flow. The model drafts a report with multi-modal interpretability and a reliability index, which the clinician refines by verifying the provided evidence. This process enhances clinical decision-making by combining AI’s analytical power with clinician expertise.
• Figure 2: a, XMedGPT consistently outperforms other generalist biomedical models across five core tasks: single-label diagnosis, multi-label diagnosis, visual question answering (VQA), multiple-choice reasoning, and image captioning. b, Overall task performance relative to model scale. c, Performance on single- and multi-label diagnosis tasks, evaluated using F1-score and micro-F1-score, respectively. d, Accuracy on the OmniMedVQA multiple-choice benchmark, covering eight medical modalities from 42 datasets. e, Comparison of closed-ended accuracy, F1-score, and open-ended recall for medical VQA; 'x' indicates missing results in the original publications. f, Benchmarking against GPT-4o and other leading GMAI models on VQA, single-label, and multi-label diagnosis tasks. g, Image captioning performance on MIMIC-CXR and IU-Xray, assessed using BLEU, ROUGE-L, and CheXpert-F1 metrics. h, Accuracy on the GMAI-MMbench multiple-choice benchmark, covering 284 datasets, 38 modalities, and 18 clinical tasks.
• Figure 3: a, c, e, Performance comparison of foundation model-based approaches and baseline methods using AUC and F1 scores. Panel a shows results for 14-month progression-free survival prediction, c for 2-year overall survival prediction in non-small cell lung cancer (NSCLC), and e for 1-year overall survival prediction in glioblastoma. Refer to the Methods section for implementation details. d,e,f,, Kaplan–Meier survival curves stratified by model-predicted risk groups across the three prognostic tasks. Risk thresholds were determined using internal validation sets to ensure fair comparison across methods.
• Figure 4: a, Comparison of our method with conventional semantic entropy (SE) and discrete entropy (DE) on four VQA tasks using AUROC to assess error prediction. b, Performance on image captioning tasks, showing our method AUC improves with more generated questions per sentence. c, Overview of curated datasets for supervised fine-tuning (10k samples) and direct preference optimization (8k samples); see \ref{['fig:Prompt_reasoning']} and \ref{['fig:Reasoning_data']} for details. d, Impact of long chain-of-thought (CoT) reasoning on clinical captioning tasks, evaluated using CheXpert F1, RadGraph F1, and CheXpert Similarity. e, Evaluation protocol for Steps 1, 2, and 4 in CoT: Step 1 and 2 assess region-level, object-level, and alignment performance; Step 4 measures completeness, consistency, and correlation between reasoning and report scores (Kendall Tau). f, DPO maintains localization accuracy in Steps 1 and 2 while improve the reasoning reliability and report accuracy. g, Improved correlation (Kendall Tau) between reasoning and report quality in Step 4 after applying DPO.
• Figure 5: a, XMedGPT is an all-in-one biomedical model capable of processing image, text, audio, and region-level inputs, enabling tasks such as region recognition, region-based VQA, and lesion localization. b, List of anatomical regions recognizable by XMedGPT, which supports localization of 141 lesion types (see \ref{['fig:page5']}b for details). c, Performance on region-related tasks: F1-score for region recognition, IoU for lesion localization, and task-specific metrics for Region VQA—acc@0.5 (ROC), SPICE (RC), recall@0.5 (VG), and mBMR (BLEU-2, METEOR, ROUGE-L) for MIA, following the BiRD convention. d, Comparative results on region recognition and lesion localization across six anatomical structures. e, Performance comparison on Region VQA tasks across eight imaging modalities. f, Overview of audio data curation and fine-tuning: 514,668 pretraining instances with audio input were used to update the audio encoder, audio MLP, and LLM (via LoRA), with other components frozen. g, Comparison of XMedGPT with audio vs. text inputs and with leading audio models on single-label diagnosis, multi-label diagnosis, and medical VQA tasks.