Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HGP-Mamba: Integrating Histology and Generated Protein Features for Mamba-based Multimodal Survival Risk Prediction

Jing Dai, Chen Wu, Ming Wu, Qibin Zhang, Zexi Wu, Jingdong Zhang, Hongming Xu

Abstract

Recent advances in multimodal learning have significantly improved cancer survival risk prediction. However, the joint prognostic potential of protein markers and histopathology images remains underexplored, largely due to the high cost and limited availability of protein expression profiling. To address this challenge, we propose HGP-Mamba, a Mamba-based multimodal framework that efficiently integrates histological with generated protein features for survival risk prediction. Specifically, we introduce a protein feature extractor (PFE) that leverages pretrained foundation models to derive high-throughput protein embeddings directly from Whole Slide Images (WSIs), enabling data-efficient incorporation of molecular information. Together with histology embeddings that capture morphological patterns, we further introduce the Local Interaction-aware Mamba (LiAM) for fine-grained feature interaction and the Global Interaction-enhanced Mamba (GiEM) to promote holistic modality fusion at the slide level, thus capture complex cross-modal dependencies. Experiments on four public cancer datasets demonstrate that HGP-Mamba achieves state-of-the-art performance while maintaining superior computational efficiency compared with existing methods. Our source code is publicly available at https://github.com/Daijing-ai/HGP-Mamba.git.

HGP-Mamba: Integrating Histology and Generated Protein Features for Mamba-based Multimodal Survival Risk Prediction

Abstract

Recent advances in multimodal learning have significantly improved cancer survival risk prediction. However, the joint prognostic potential of protein markers and histopathology images remains underexplored, largely due to the high cost and limited availability of protein expression profiling. To address this challenge, we propose HGP-Mamba, a Mamba-based multimodal framework that efficiently integrates histological with generated protein features for survival risk prediction. Specifically, we introduce a protein feature extractor (PFE) that leverages pretrained foundation models to derive high-throughput protein embeddings directly from Whole Slide Images (WSIs), enabling data-efficient incorporation of molecular information. Together with histology embeddings that capture morphological patterns, we further introduce the Local Interaction-aware Mamba (LiAM) for fine-grained feature interaction and the Global Interaction-enhanced Mamba (GiEM) to promote holistic modality fusion at the slide level, thus capture complex cross-modal dependencies. Experiments on four public cancer datasets demonstrate that HGP-Mamba achieves state-of-the-art performance while maintaining superior computational efficiency compared with existing methods. Our source code is publicly available at https://github.com/Daijing-ai/HGP-Mamba.git.
Paper Structure (19 sections, 9 equations, 5 figures, 2 tables)

This paper contains 19 sections, 9 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Space State Models (SSMs)
  4. Multi-modal Feature Extraction
  5. Histology features
  6. Protein features
  7. Local Interaction-aware Mamba (LiAM)
  8. Golobal Interaction-enhanced Mamba (GiEM)
  9. Survival Risk Prediction
  10. Experiments
  11. Datasets and Metrics
  12. Implementation Details
  13. Comparison Results
  14. Patient Stratification
  15. Ablation Studies
  16. ...and 4 more sections

Figures (5)

  • Figure 1: HGP-Mamba overview. (a) Details of the proposed HGP-Mamba which is consisted of three steps: multi-modal feature extraction, feature interaction and enhancement and risk prediction. (b) Schematic of the Local Interaction-aware Mamba (LiAM). (c) Architecture of the Global Interaction-enhanced Mamba (GiEM).
  • Figure 2: Illustration of our protein feature extractor (PFE). Note that ConvNet is the backbone of the ROISE model.
  • Figure 3: Kaplan-Meier survival curves of the proposed model on four cancer datasets.
  • Figure 4: (a) Comparison of different multimodal fusion methods. (b) Inference time comparison with varying patches
  • Figure 5: Spatial expression heatmaps for PD-L1 on randomly selected slides from the TCGA-COADREAD dataset. For each sample, the left panel shows the WSI thumbnail, the middle panel overlays the predicted PD-L1 expression heatmap on the WSI, and the right panel displays the selected patches according to the predicted expression level.