Cohort-Individual Cooperative Learning for Multimodal Cancer Survival Analysis

TL;DR

The paper tackles cancer survival analysis with multimodal data (genomics and pathology) by addressing heterogeneity and high dimensionality through Cohort-individual Cooperative Learning (CCL). It introduces Multimodal Knowledge Decomposition (MKD) to explicitly separate redundancy, synergy, and modality-specific knowledge (G, P, C, S) and Cohort Guidance Modeling (CGM) to enhance generalization at both knowledge and patient levels via cohort contrastive learning. A Transformer-based fusion then combines the four knowledge components to predict survival, with a censorship-aware loss and a cohort-regularized term. The approach achieves state-of-the-art C-index scores on five TCGA datasets, showing strong discrimination and generalization across cancers. This framework has potential to improve robustness of multimodal survival models and guide future integration of additional modalities and cohort-aware supervision; $L = L_{surv} + \alpha L_{cohort}$ encapsulates the joint objective guiding learning.

Abstract

Recently, we have witnessed impressive achievements in cancer survival analysis by integrating multimodal data, e.g., pathology images and genomic profiles. However, the heterogeneity and high dimensionality of these modalities pose significant challenges for extracting discriminative representations while maintaining good generalization. In this paper, we propose a Cohort-individual Cooperative Learning (CCL) framework to advance cancer survival analysis by collaborating knowledge decomposition and cohort guidance. Specifically, first, we propose a Multimodal Knowledge Decomposition (MKD) module to explicitly decompose multimodal knowledge into four distinct components: redundancy, synergy and uniqueness of the two modalities. Such a comprehensive decomposition can enlighten the models to perceive easily overlooked yet important information, facilitating an effective multimodal fusion. Second, we propose a Cohort Guidance Modeling (CGM) to mitigate the risk of overfitting task-irrelevant information. It can promote a more comprehensive and robust understanding of the underlying multimodal data, while avoiding the pitfalls of overfitting and enhancing the generalization ability of the model. By cooperating the knowledge decomposition and cohort guidance methods, we develop a robust multimodal survival analysis model with enhanced discrimination and generalization abilities. Extensive experimental results on five cancer datasets demonstrate the effectiveness of our model in integrating multimodal data for survival analysis.

Figures (7)

• Figure 1: Cohort knowledge offers a global view of multimodal data, assisting deep models to capture general multimodal interactions and facilitating a more effective fusion.
• Figure 2: An overview of the proposed Cohort-individual Cooperative Learning (CCL) strategy that learns comprehensive and discriminative knowledge components from multimodal data under the cohort guidance. Our CCL includes 1) a Multimodal Knowledge Decomposition (MKD) module to comprehensively and explicitly decompose heterogeneous multimodal knowledge; and 2) a Cohort Guidance Modeling (CGM) to learn more general and discriminative knowledge components.
• Figure 3: Cluster center alignment for pathology features. Aligned centers at the same positions typically exhibit similar phenotypes across patients, enabling the network to extract specific information about these phenotypes.
• Figure 4: The structure of common and synergistic encoders. We generate attention vectors based on the co-attention matrix to integrate multimodal features. The learnable parameters for common and synergistic knowledge are different, enabling the same structure used to extract distinct knowledge components. The $*$ and $+$ indicate element-wise multiplication and addition, respectively.
• Figure 5: Graphical illustration of cohort guidance. Colors and shapes indicate knowledge components and patient groups, respectively.