Handling Missing Modalities in Multimodal Survival Prediction for Non-Small Cell Lung Cancer

TL;DR

This work tackles survival prediction in Non-Small Cell Lung Cancer under realistic missingness by introducing a missing-aware multimodal framework that fuses CT, Whole-Slide Histopathology, and structured clinical data using foundation-model features and a Not-Another-Imputation-Method (NAIM) with Oblivious Differentiable Survival Tree (ODST) heads. The proposed three-stage pipeline—modality-specific FM feature extraction, missing-aware unimodal encoding, and intermediate fusion—enables robust survival modeling without imputing or discarding incomplete cases. Empirical results show that intermediate fusion with WSI and Tabular data delivers the strongest prognostic performance (C-index ≈ 0.733), and the model provides clinically meaningful risk stratification for overall survival and disease progression events. The findings underscore the practical potential of missing-native multimodal learning for oncology, while highlighting the importance of modality complementarity and adaptive weighting in small, sparse cohorts; the authors also release their curated dataset to foster reproducibility and further research in this area.

Abstract

Accurate survival prediction in Non-Small Cell Lung Cancer (NSCLC) requires the integration of heterogeneous clinical, radiological, and histopathological information. While Multimodal Deep Learning (MDL) offers a promises for precision prognosis and survival prediction, its clinical applicability is severely limited by small cohort sizes and the presence of missing modalities, often forcing complete-case filtering or aggressive imputation. In this work, we present a missing-aware multimodal survival framework that integrates Computed Tomography (CT), Whole-Slide Histopathology (WSI) Images, and structured clinical variables for overall survival modeling in unresectable stage II-III NSCLC. By leveraging Foundation Models (FM) for modality-specific feature extraction and a missing-aware encoding strategy, the proposed approach enables intermediate multimodal fusion under naturally incomplete modality profiles. The proposed architecture is resilient to missing modalities by design, allowing the model to utilize all available data without being forced to drop patients during training or inference. Experimental results demonstrate that intermediate fusion consistently outperforms unimodal baselines as well as early and late fusion strategies, with the strongest performance achieved by the fusion of WSI and clinical modalities (73.30 C-index). Further analyses of modality importance reveal an adaptive behavior in which less informative modalities, i.e., CT modality, are automatically down-weighted and contribute less to the final survival prediction.

Figures (6)

• Figure 1: Overview of modality availability and clinical data sparsity in the cohort. Panel a reports the availability of entire diagnostic modalities (WSI, CT and Tabular data) across the cohort. Panel b presents the percentage of missing values that characterize the Tabular modality, detailing missingness rates for each individual features.
• Figure 2: Schematic overview of the proposed Multimodal Survival Framework. The architecture integrates three distinct clinical data streams: Pathology (WSI), Radiology (CT), and Tabular Clinical data. Step 1: Preprocessing & Feature Extraction. High level representations are extracted using domain-specific FMs. Whole Slide Images are processed via CLAM and encoded using the WSI patch extraction pipeline; volumetric CT scans are encoded using the Merlin 3D ViT encoder. Step 2: Missing-Aware Encoding. Each modality stream is processed by a dedicated NAIM+ODST encoder. This transformer-based encoder employs an adaptive masking mechanism to dynamically handle missing modalities without imputation. Step 3: Intermediate Fusion & Prediction. The diagram visualizes the fusion paradigms. In the proposed intermediate fusion, latent representations from the NAIM+ODST encoders are concatenated and fed into an ODST layer to predict the output patient-specific hazard function $y$.
• Figure 3: 2D UMAP projections derived from FM embeddings for: a) WSI, b) CT, and c) Tabular. Each plot displays the distribution of Survivors (green data points) and Non-survivors (red data points), including kernel density contours and class centroids. The annotation in each panel reports the p-value of the group separation test computed on the embedding space.
• Figure 4: Performance under increasing missingness for multimodal intermediate fusion combinations. Each panel reports the mean C-index $\pm$ Standard Error over 5 folds. (a) WSI+CT+Tab model. (b) WSI+CT model. (c) WSI+Tabular model. (d) CT+Tabular model.
• Figure 5: Stratified Kaplan--Meier survival curves for intermediate fusion models (ConcatODST), with follow-up time (days) on the x-axis and survival probability on the y-axis. Patients are stratified into Low-Risk and High-Risk groups according to the predicted 5-year outcome score produced by each model. Panels a-d report the resulting risk-stratified curves across different modality combinations.