Multimodal Feature Prototype Learning for Interpretable and Discriminative Cancer Survival Prediction

TL;DR

FeatProto tackles the interpretability gap in cancer survival analysis by integrating whole-slide pathology and genomic data into a unified feature prototype space. It introduces EMA ProtoUp for stable cross-modal prototype evolution, Wandering Prototypes to capture tumor heterogeneity, and MPMatch for robust multilevel prototype matching, all supervised by a ProtoSurv Loss that aligns prototype reasoning with survival outcomes. Empirical results across four TCGA cohorts show superior predictive performance over state-of-the-art unimodal and multimodal approaches, with transparent decision pathways traced via a Prototype Source Library. The approach advances clinical applicability by delivering accurate, traceable prognostic insights and sets the stage for extensions to personalized oncology and real-time prognostic monitoring.

Abstract

Survival analysis plays a vital role in making clinical decisions. However, the models currently in use are often difficult to interpret, which reduces their usefulness in clinical settings. Prototype learning presents a potential solution, yet traditional methods focus on local similarities and static matching, neglecting the broader tumor context and lacking strong semantic alignment with genomic data. To overcome these issues, we introduce an innovative prototype-based multimodal framework, FeatProto, aimed at enhancing cancer survival prediction by addressing significant limitations in current prototype learning methodologies within pathology. Our framework establishes a unified feature prototype space that integrates both global and local features of whole slide images (WSI) with genomic profiles. This integration facilitates traceable and interpretable decision-making processes. Our approach includes three main innovations: (1) A robust phenotype representation that merges critical patches with global context, harmonized with genomic data to minimize local bias. (2) An Exponential Prototype Update Strategy (EMA ProtoUp) that sustains stable cross-modal associations and employs a wandering mechanism to adapt prototypes flexibly to tumor heterogeneity. (3) A hierarchical prototype matching scheme designed to capture global centrality, local typicality, and cohort-level trends, thereby refining prototype inference. Comprehensive evaluations on four publicly available cancer datasets indicate that our method surpasses current leading unimodal and multimodal survival prediction techniques in both accuracy and interoperability, providing a new perspective on prototype learning for critical medical applications. Our source code is available at https://github.com/JSLiam94/FeatProto.

Figures (10)

• Figure 1: Schematic of survival prediction. (a) Multimodal feature embedding & fusion; (b) Conventional survival prediction (decision head: linear fully-connected layer); (c) Proposed feature prototype learning: prototype library with global-local fused features and multilevel deep prototype matching for accurate, interpretable survival prediction; (d) Traditional prototype library (local image only).
• Figure 2: Prototype Library Construction and Update. Left: Prototype library construction. The initial model generates feature embeddings to derive a similarity matrix, which is clustered to form a central prototype library. Feature prototypes are categorized by survival risk levels. Middle: EMA ProtoUp. Old prototypes migrate toward new features via EMA-based updates to generate new prototypes, representing typical samples. Right: Design and update mechanism of Wandering Prototypes. During prototype library construction or update, edge cases are selected from new features to represent special samples.
• Figure 3: Demonstration of the multilevel deep prototype matching strategy. Through weighted fusion of center similarity, class average similarity, and nearest prototype similarity, it synergistically optimizes local feature alignment and global class guidance. Meanwhile, through similarity visualization, it further enhances the interpretability of the model.
• Figure 4: Schematic of ProtoSurv Loss: ProtoSurv Loss creates a prototype space focusing on intra-class convergence and inter-class separation, while NLLSurvLoss uses risk score stratification. Their combined approach enhances updates to the prototype library and survival analysis.
• Figure 5: Kaplan-Meier survival curves and box plots of risk score distribution on the LUAD and BLCA datasets.