BAAI Cardiac Agent: An intelligent multimodal agent for automated reasoning and diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging

Abstract

Cardiac magnetic resonance (CMR) is a cornerstone for diagnosing cardiovascular disease. However, it remains underutilized due to complex, time-consuming interpretation across multi-sequences, phases, quantitative measures that heavily reliant on specialized expertise. Here, we present BAAI Cardiac Agent, a multimodal intelligent system designed for end-to-end CMR interpretation. The agent integrates specialized cardiac expert models to perform automated segmentation of cardiac structures, functional quantification, tissue characterization and disease diagnosis, and generates structured clinical reports within a unified workflow. Evaluated on CMR datasets from two hospitals (2413 patients) spanning 7-types of major cardiovascular diseases, the agent achieved an area under the receiver-operating-characteristic curve exceeding 0.93 internally and 0.81 externally. In the task of estimating left ventricular function indices, the results generated by this system for core parameters such as ejection fraction, stroke volume, and left ventricular mass are highly consistent with clinical reports, with Pearson correlation coefficients all exceeding 0.90. The agent outperformed state-of-the-art models in segmentation and diagnostic tasks, and generated clinical reports showing high concordance with expert radiologists (six readers across three experience levels). By dynamically orchestrating expert models for coordinated multimodal analysis, this agent framework enables accurate, efficient CMR interpretation and highlights its potentials for complex clinical imaging workflows. Code is available at https://github.com/plantain-herb/Cardiac-Agent.

Figures (11)

• Figure 1: A schematic overview of the BAAI Cardiac Agent illustrating the process of radiology report generation from multi-sequence CMR scans (upper section), in comparison with the traditional clinical workflow (lower section). Upper section: the agent-assisted workflow, in which the user submits instructions and upload all CMR sequences to the BAAI Cardiac Agent. The agent automatically interprets the request, invokes appropriate analytic modules, and performs segmentation, quantitative assessment, and diagnostic reasoning. A structured, quantitative enriched radiology report is then generated within approximately 90 seconds. Lower section: the workflow of a traditional radiologist. This workflow begins with manual or semi-automatic segmentation of cine images, followed by qualification of key cardiac function parameters. Diagnostic interpretation is then performed by integrating information from late gadolinium enhancement (LGE) and rest myocardium perfusion imaging (Rest MPI), after which the final report is manually compiled. This sequential workflow typically requires approximately 1800 seconds.
• Figure 2: Performance of the expert CVDs diagnostic model in internal and external testing. a–c: Receiver operating characteristic curves for CDS; d–h: Receiver operating characteristic curves for NICMS. The red curves represent the internal test set, and the blue curves represent the external validation set. i: Confusion matrix of the AI diagnostic model predictions versus ground truth labels based on the full cohort (n = 2,232), with a color gradient visually representing the proportional distribution of predictions across disease categories.
• Figure 3: Evaluation results of BAAI Cardiac Agent report consistency and expert model invocation success rate. (a) Bland-Altman plots comparing BAAI Cardiac Agent measurements (left ventricular end-diastolic volume, LVEDV; left ventricular end-systolic volume, LVESV; left ventricular ejection fraction, LVEF; stroke volume, SV; left ventricular mass, LVM; left ventricular end-diastolic diameter, LVEDD) with manual reports in the internal cohort; (b) Mean distribution of left ventricular end-diastolic wall thickness (LVEDWT) measured by manual reports using the 17-SM and by the BAAI Cardiac Agent in the internal cohort. The bullseye plot correlates the basal, mid, and apical segments of the left ventricle with the outer, middle, and inner layers in sequence, with the apex at the center; (c) Success rate of expert model invocations by the BAAI Cardiac Agent on the internal and external test sets; (d) Accuracy performance of the BAAI Cardiac Agent in cardiac-specific VQA tasks.
• Figure 4: Schematic diagram of the BAAI Cardiac Agent workflow, in which assigned tasks are completed through adaptive invocation and coordinated use of specialized medical image segmentation tools.
• Figure 5: Task-modality-tool mapping of the BAAI Cardiac Agent: the left panel presents the tasks integrated in the agent and their corresponding medical imaging modalities, while the right panel lists the tools required to complete the respective imaging tasks.