Multimodal Cancer Survival Analysis via Hypergraph Learning with Cross-Modality Rebalance

TL;DR

This work addresses the core challenge of multimodal cancer survival prediction by combining high-resolution pathology (WSIs) with genomics. It introduces MRePath, a framework that uses a sheaf-enabled hypergraph to capture contextual and hierarchical information in WSIs and a dynamic modality rebalance with cross-modal fusion to mitigate pathology-genomics imbalance. Across five TCGA cohorts, MRePath achieves a mean C-Index of approximately 71.5%, outperforming state-of-the-art methods by about 3.4 percentage points, and its ablations demonstrate the effectiveness of jointly modeling high-order patch relationships and adaptive, interactive fusion. The approach advances prognostic accuracy and interpretability by illustrating modality interactions and identifying gene signatures associated with risk, with potential clinical impact for more reliable survival predictions.

Abstract

Multimodal pathology-genomic analysis has become increasingly prominent in cancer survival prediction. However, existing studies mainly utilize multi-instance learning to aggregate patch-level features, neglecting the information loss of contextual and hierarchical details within pathology images. Furthermore, the disparity in data granularity and dimensionality between pathology and genomics leads to a significant modality imbalance. The high spatial resolution inherent in pathology data renders it a dominant role while overshadowing genomics in multimodal integration. In this paper, we propose a multimodal survival prediction framework that incorporates hypergraph learning to effectively capture both contextual and hierarchical details from pathology images. Moreover, it employs a modality rebalance mechanism and an interactive alignment fusion strategy to dynamically reweight the contributions of the two modalities, thereby mitigating the pathology-genomics imbalance. Quantitative and qualitative experiments are conducted on five TCGA datasets, demonstrating that our model outperforms advanced methods by over 3.4\% in C-Index performance.

Figures (7)

• Figure 1: Left: Compared to MIL, hypergraph learning activates more patch regions and better captures contextual and hierarchical details. Right: Examples reveal the pathology-genomics imbalance, where pathology features dominate the overall survival prediction.
• Figure 2: Overview of MRePath.A, MRePath consists of feature extraction from pathology and genomics modalities, hypergraph learning for capturing WSI representations, and modality rebalance including dynamic weighting and interactive alignment fusion for recalibrating two modalities. B, Hypergraph learning involves constructing topological-based and feature-based hyperedges, and employing sheaf hypergraph to encourage local and global interactions among hyperedges, thereby capturing richer contextual and hierarchical details. C, Dynamic weighting computes mono-confidence and holo-confidence to produce modality weights for rebalancing modality contributions.
• Figure 3: Ablation study on hyperedge construction threshold $k$. Performance of various similarity thresholds ($k$ = 0, 5, 9, 25, 49) is reported in C-Index ($\%$).
• Figure 4: Visualization for low and high-risk cases in the BRCA (Top): The heatmaps are generated using cross-attention scores, with red and blue for high and low scores. The top five most influential genes are also highlighted in red for high and blue for low. Kaplan-Meier curves (Bottom): The high-risk and low-risk groups are determined based on the median predicted risk scores of our model. Our model achieved p-values less than 0.05 across all five datasets, demonstrating its excellent stratification capability.
• Figure 5: Visualization for the distribution of each dataset using box plots based on censorship status. Black diamonds represent outliers.