SurvAgent: Hierarchical CoT-Enhanced Case Banking and Dichotomy-Based Multi-Agent System for Multimodal Survival Prediction

TL;DR

SurvAgent addresses the need for transparent multimodal survival prediction in oncology by coupling a WSI-Gene CoT-enhanced case banking system with a dichotomy-based multi-expert inference stage. It constructs two CoT-driven case banks (WSI and gene) that store full reasoning traces and allow experiential learning, then retrieves similar cases and integrates multimodal reports with expert predictions through progressive interval refinement. The approach leverages hierarchical WSI analysis (LMScreen, CoSMining, ConfMining) and gene categorization across six functional types, yielding interpretable CoT explanations for each prediction. Across five TCGA cohorts, SurvAgent achieves state-of-the-art C-index and robust patient stratification while providing transparent reasoning workflows, offering a practical pathway toward clinically trusted AI-assisted survival prognosis.

Abstract

Survival analysis is critical for cancer prognosis and treatment planning, yet existing methods lack the transparency essential for clinical adoption. While recent pathology agents have demonstrated explainability in diagnostic tasks, they face three limitations for survival prediction: inability to integrate multimodal data, ineffective region-of-interest exploration, and failure to leverage experiential learning from historical cases. We introduce SurvAgent, the first hierarchical chain-of-thought (CoT)-enhanced multi-agent system for multimodal survival prediction. SurvAgent consists of two stages: (1) WSI-Gene CoT-Enhanced Case Bank Construction employs hierarchical analysis through Low-Magnification Screening, Cross-Modal Similarity-Aware Patch Mining, and Confidence-Aware Patch Mining for pathology images, while Gene-Stratified analysis processes six functional gene categories. Both generate structured reports with CoT reasoning, storing complete analytical processes for experiential learning. (2) Dichotomy-Based Multi-Expert Agent Inference retrieves similar cases via RAG and integrates multimodal reports with expert predictions through progressive interval refinement. Extensive experiments on five TCGA cohorts demonstrate SurvAgent's superority over conventional methods, proprietary MLLMs, and medical agents, establishing a new paradigm for explainable AI-driven survival prediction in precision oncology.

Figures (12)

• Figure 1: SurvAgent utilizes WSI-Gene CoT-Enhanced Case Banks through hierarchical WSI analysis and gene-stratified analysis, then performs dichotomy-based multi-expert inference by retrieving cases and progressively refining survival predictions.
• Figure 1: Overview of the WSI Attribute Checklist
• Figure 2: Overview of SurvAgent. (1) WSI-Gene CoT-Enhanced Case Bank Construction includes Hierarchical WSI CoT-Enhanced Case Bank that progressively analyzes WSIs at multiple magnifications through LMScreen, CoSMining, and ConfMining, and Gene-Stratified CoT-Enhanced Case Bank for gene statistical analysis. PathAgent and GenAgent generate structured reports and CoT reasoning with self-critique for their respective case banks. (2) Dichotomy-Based Multi-Expert Agent Inference uses RAG for retrieval and integrates retrieved cases, reports, and expert predictions for progressive survival time prediction from coarse to fine-grained intervals.
• Figure 2: Case studies of SurvAgent, including WSI Summarized report, Gene Summarized report, Final Result, and Survival Time gt (Case ID: TCGA-XF-A9SJ).
• Figure 3: Kaplan-Meier Analysis of predicted high-risk (red) and low-risk (blue) groups on five cancer datasets and their p-values. Shaded areas refer to the confidence intervals.