CardioCoT: Hierarchical Reasoning for Multimodal Survival Analysis

TL;DR

CardioCoT tackles recurrence risk of major adverse cardiovascular events after acute myocardial infarction by leveraging postoperative cardiac MRI and clinical notes. It introduces a two-stage hierarchical reasoning framework: stage 1 generates evidence-augmented reasoning trajectories guided by radiological findings, and stage 2 fuses these trajectories with imaging data to build a survival predictor. The approach employs an iterative Thinker-Oracle self-refinement loop with limited iterations, followed by LLM/VLM fine-tuning and multimodal fusion via attention, achieving a 7.53% improvement in C-index on a real-world dataset and significantly better Kaplan-Meier separation, while providing interpretable reasoning pathways for clinical decision support. This work advances interpretability and predictive accuracy in multimodal survival analysis and suggests broader applicability to cardiovascular decision-support tasks.

Abstract

Accurate prediction of major adverse cardiovascular events recurrence risk in acute myocardial infarction patients based on postoperative cardiac MRI and associated clinical notes is crucial for precision treatment and personalized intervention. Existing methods primarily focus on risk stratification capability while overlooking the need for intermediate robust reasoning and model interpretability in clinical practice. Moreover, end-to-end risk prediction using LLM/VLM faces significant challenges due to data limitations and modeling complexity. To bridge this gap, we propose CardioCoT, a novel two-stage hierarchical reasoning-enhanced survival analysis framework designed to enhance both model interpretability and predictive performance. In the first stage, we employ an evidence-augmented self-refinement mechanism to guide LLM/VLMs in generating robust hierarchical reasoning trajectories based on associated radiological findings. In the second stage, we integrate the reasoning trajectories with imaging data for risk model training and prediction. CardioCoT demonstrates superior performance in MACE recurrence risk prediction while providing interpretable reasoning processes, offering valuable insights for clinical decision-making.

Figures (6)

• Figure 1: Comparison of LLM/LLM utilization in survival analysis: 1) Prompt-based: risk estimation via direct survival prompt; 2) Format-based: structured covariates extraction from unstructured data; 3) Embedding-based: semantic embedding extraction from pre-trained LLM/VLM; 4) Ours: hierarchical reasoning enabling evidential support and interpretable analysis.
• Figure 2: The proposed CardioCoT consists of two stages: (1) GPT-4o generates hierarchical evidence-augmented reasoning with self-refinement, fine-tuned on advanced LLMs/VLMs; (2) MRI scans and reasoning outputs are encoded by vision and text encoders, fused via an attention module for final risk prediction.
• Figure 2: Ablation with reasoning-enhanced diagnosis (D.), complications (C.), and MACE follow-up (M.).
• Figure 3: The KM analysis curves results.
• Figure 4: Attention quantification of different parts for survival analysis.