Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MUST: Modality-Specific Representation-Aware Transformer for Diffusion-Enhanced Survival Prediction with Missing Modality

Kyungwon Kim, Dosik Hwang

Abstract

Accurate survival prediction from multimodal medical data is essential for precision oncology, yet clinical deployment faces a persistent challenge: modalities are frequently incomplete due to cost constraints, technical limitations, or retrospective data availability. While recent methods attempt to address missing modalities through feature alignment or joint distribution learning, they fundamentally lack explicit modeling of the unique contributions of each modality as opposed to the information derivable from other modalities. We propose MUST (Modality-Specific representation-aware Transformer), a novel framework that explicitly decomposes each modality's representation into modality-specific and cross-modal contextualized components through algebraic constraints in a learned low-rank shared subspace. This decomposition enables precise identification of what information is lost when a modality is absent. For the truly modality-specific information that cannot be inferred from available modalities, we employ conditional latent diffusion models to generate high-quality representations conditioned on recovered shared information and learned structural priors. Extensive experiments on five TCGA cancer datasets demonstrate that MUST achieves state-of-the-art performance with complete data while maintaining robust predictions in both missing pathology and missing genomics conditions, with clinically acceptable inference latency.

MUST: Modality-Specific Representation-Aware Transformer for Diffusion-Enhanced Survival Prediction with Missing Modality

Abstract

Accurate survival prediction from multimodal medical data is essential for precision oncology, yet clinical deployment faces a persistent challenge: modalities are frequently incomplete due to cost constraints, technical limitations, or retrospective data availability. While recent methods attempt to address missing modalities through feature alignment or joint distribution learning, they fundamentally lack explicit modeling of the unique contributions of each modality as opposed to the information derivable from other modalities. We propose MUST (Modality-Specific representation-aware Transformer), a novel framework that explicitly decomposes each modality's representation into modality-specific and cross-modal contextualized components through algebraic constraints in a learned low-rank shared subspace. This decomposition enables precise identification of what information is lost when a modality is absent. For the truly modality-specific information that cannot be inferred from available modalities, we employ conditional latent diffusion models to generate high-quality representations conditioned on recovered shared information and learned structural priors. Extensive experiments on five TCGA cancer datasets demonstrate that MUST achieves state-of-the-art performance with complete data while maintaining robust predictions in both missing pathology and missing genomics conditions, with clinically acceptable inference latency.

Paper Structure

This paper contains 33 sections, 22 equations, 10 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multimodal Survival Prediction
  4. Handling Missing Modalities
  5. Diffusion Models in Medical Imaging
  6. Method
  7. Problem Formulation
  8. Data Preparation and Feature Extraction
  9. Multimodal Algebraic Decomposition
  10. Progressive Training Strategy
  11. Latent Diffusion Model for Missing Modality
  12. Inference with Missing Modality
  13. Experiments
  14. Datasets and Implementation Details
  15. Comparison with State-of-the-Art Methods
  16. ...and 18 more sections

Figures (10)

  • Figure 1: Overall architecture of MUST. The framework extracts global representations $g_P$ and $g_G$ via self-attention, computes shared information through bidirectional cross-attention, and decomposes each modality into specific and shared components for survival prediction.
  • Figure 2: Conditional Diffusion Model
  • Figure 3: Kaplan-Meier survival curves on BLCA dataset comparing high-risk and low-risk groups: (a) complete data, (b) missing $G$, (c) missing $P$. Shaded regions represent 95% confidence intervals.
  • Figure 4: Ablation on rank $r$ of the shared subspace $P_\cap$.
  • Figure 5: ECDF curve and cosine similarity map on UCEC dataset.
  • ...and 5 more figures