A Flow Model with Low-Rank Transformers for Incomplete Multimodal Survival Analysis

TL;DR

This work tackles incomplete multimodal survival analysis by introducing a distribution-consistent framework that fuses a class-specific normalizing flow for cross-modal alignment with a low-rank Transformer for efficient intra-modal modeling. The approach enables robust survival prediction even when one modality is missing, through bidirectional flow mappings, class-conditioned latent spaces, and a lightweight attention mechanism. Empirical results on five TCGA cohorts show state-of-the-art performance under both complete and incomplete data, with strong risk stratification and favorable computational efficiency. Overall, the method offers practical robustness and interpretability, suitable for real-world clinical deployment where modality dropout is common.

Abstract

In recent years, multimodal medical data-based survival analysis has attracted much attention. However, real-world datasets often suffer from the problem of incomplete modality, where some patient modality information is missing due to acquisition limitations or system failures. Existing methods typically infer missing modalities directly from observed ones using deep neural networks, but they often ignore the distributional discrepancy across modalities, resulting in inconsistent and unreliable modality reconstruction. To address these challenges, we propose a novel framework that combines a low-rank Transformer with a flow-based generative model for robust and flexible multimodal survival prediction. Specifically, we first formulate the concerned problem as incomplete multimodal survival analysis using the multi-instance representation of whole slide images (WSIs) and genomic profiles. To realize incomplete multimodal survival analysis, we propose a class-specific flow for cross-modal distribution alignment. Under the condition of class labels, we model and transform the cross-modal distribution. By virtue of the reversible structure and accurate density modeling capabilities of the normalizing flow model, the model can effectively construct a distribution-consistent latent space of the missing modality, thereby improving the consistency between the reconstructed data and the true distribution. Finally, we design a lightweight Transformer architecture to model intra-modal dependencies while alleviating the overfitting problem in high-dimensional modality fusion by virtue of the low-rank Transformer. Extensive experiments have demonstrated that our method not only achieves state-of-the-art performance under complete modality settings, but also maintains robust and superior accuracy under the incomplete modalities scenario.

Figures (4)

• Figure 1: Comparison of multimodal survival prediction frameworks. (a) Traditional approaches encode each modality independently and perform direct fusion for survival prediction, without explicitly modeling modality correlations. (b) Conditional VAE-based methods reconstruct missing modalities using conditional priors but may suffer from distributional inconsistency between the training and testing stages. (c) Our proposed distribution-consistent framework introduces bidirectional flow-based transformation to align modality distributions across missing and complete settings, enabling robust and consistent prediction even under modality dropout.
• Figure 2: Overview of our distribution-consistent multimodal survival prediction framework. Image and gene features are extracted via modality-specific encoders. Visual features are modeled with normalizing flows to learn a latent distribution $\mathcal{N}(\mu_z, \Sigma_z)$, while gene features are aligned via cross-modal distribution transfer and reconstructed through decoder $D(\cdot)$ if needed. The fused representations are fed into a Low-rank Transformer for final survival prediction.
• Figure 3: Quantitative and ablation analysis on the LUAD and BLCA datasets. (a) Kaplan–Meier survival curves for predicted high-risk and low-risk groups on the LUAD dataset, with a statistically significant separation. (b) Ablation study results showing the impact of removing key components on the C-index for LUAD and BLCA. (c) Comparison of computational efficiency versus predictive performance on LUAD.
• Figure 4: Visualization of low-rank attention under the missing modality setting. From left to right: the original whole-slide image (WSI), the corresponding low-rank attention map highlighting tumor-suppressed regions, and the top-5 histopathological patches with the highest low-rank attention scores.