Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SELECTOR: Heterogeneous graph network with convolutional masked autoencoder for multimodal robust prediction of cancer survival

Liangrui Pan, Yijun Peng, Yan Li, Xiang Wang, Wenjuan Liu, Liwen Xu, Qingchun Liang, Shaoliang Peng

Abstract

Accurately predicting the survival rate of cancer patients is crucial for aiding clinicians in planning appropriate treatment, reducing cancer-related medical expenses, and significantly enhancing patients' quality of life. Multimodal prediction of cancer patient survival offers a more comprehensive and precise approach. However, existing methods still grapple with challenges related to missing multimodal data and information interaction within modalities. This paper introduces SELECTOR, a heterogeneous graph-aware network based on convolutional mask encoders for robust multimodal prediction of cancer patient survival. SELECTOR comprises feature edge reconstruction, convolutional mask encoder, feature cross-fusion, and multimodal survival prediction modules. Initially, we construct a multimodal heterogeneous graph and employ the meta-path method for feature edge reconstruction, ensuring comprehensive incorporation of feature information from graph edges and effective embedding of nodes. To mitigate the impact of missing features within the modality on prediction accuracy, we devised a convolutional masked autoencoder (CMAE) to process the heterogeneous graph post-feature reconstruction. Subsequently, the feature cross-fusion module facilitates communication between modalities, ensuring that output features encompass all features of the modality and relevant information from other modalities. Extensive experiments and analysis on six cancer datasets from TCGA demonstrate that our method significantly outperforms state-of-the-art methods in both modality-missing and intra-modality information-confirmed cases. Our codes are made available at https://github.com/panliangrui/Selector.

SELECTOR: Heterogeneous graph network with convolutional masked autoencoder for multimodal robust prediction of cancer survival

Abstract

Accurately predicting the survival rate of cancer patients is crucial for aiding clinicians in planning appropriate treatment, reducing cancer-related medical expenses, and significantly enhancing patients' quality of life. Multimodal prediction of cancer patient survival offers a more comprehensive and precise approach. However, existing methods still grapple with challenges related to missing multimodal data and information interaction within modalities. This paper introduces SELECTOR, a heterogeneous graph-aware network based on convolutional mask encoders for robust multimodal prediction of cancer patient survival. SELECTOR comprises feature edge reconstruction, convolutional mask encoder, feature cross-fusion, and multimodal survival prediction modules. Initially, we construct a multimodal heterogeneous graph and employ the meta-path method for feature edge reconstruction, ensuring comprehensive incorporation of feature information from graph edges and effective embedding of nodes. To mitigate the impact of missing features within the modality on prediction accuracy, we devised a convolutional masked autoencoder (CMAE) to process the heterogeneous graph post-feature reconstruction. Subsequently, the feature cross-fusion module facilitates communication between modalities, ensuring that output features encompass all features of the modality and relevant information from other modalities. Extensive experiments and analysis on six cancer datasets from TCGA demonstrate that our method significantly outperforms state-of-the-art methods in both modality-missing and intra-modality information-confirmed cases. Our codes are made available at https://github.com/panliangrui/Selector.
Paper Structure (25 sections, 13 equations, 5 figures, 2 tables, 1 algorithm)

This paper contains 25 sections, 13 equations, 5 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multimodal Cancer Survival Prediction
  4. Heterogeneous graph convolutional neural network
  5. Masked autoencoder
  6. Methods
  7. Overview
  8. Construction of multimodal heterogeneous graphs
  9. Feature edge reconstruction
  10. Convolutional masked autoencoder
  11. Feature cross fusion
  12. Strategies for model prediction of patient survival
  13. Experiments
  14. Datasets
  15. Experiments details
  16. ...and 10 more sections

Figures (5)

  • Figure 1: $a$: Flowchart of learning cancer multimodal datasets to predict patient survival using a heterogeneous graph-aware network based on generative encoders. $b$: Flowchart of feature edge reconstruction module based on multimodal graph. $c$: Workflow of fully convolution-based masked autoencoder. $d$: 7 schemes for multimodal prediction of patient survival.
  • Figure 2: Comparison of the optimal method of KM analysis and SELECTOR in six cancer datasets. For each dataset, patients were divided into high-risk groups (red zone) and low-risk groups (green zone) based on the median score output of the prediction model. The $P$ value for each Log-rank test is placed in the corner of each figure. $a$ shows the best model (HGCN) to obtain survival predictions. $b$ shows the survival prediction obtained by SELECTOR.
  • Figure 3: Explanation of the SELECTOR framework on the LUAD dataset. $a$: Pathological section heat map. $b$: Concern value of clinical records. $c$: Comprehensive gradient analysis of genetic maps.
  • Figure 4: Models containing different graph convolutions predict survival and obtain C-index radar plot of ablation experiment results. $P$ represents pathological modality, $G$ represents gene expression modality, $C$ represents clinical data, and & represents a combination of modalities.
  • Figure 5: Radar plot of ablation experiment results for C-index obtained by model prediction survival with different mask decoders. $P$ represents pathological modality, $G$ represents gene expression modality, $C$ represents clinical data, and & represents a combination of modalities.