ICFNet: Integrated Cross-modal Fusion Network for Survival Prediction

TL;DR

This work addresses survival prediction by integrating multi-modal patient data, including histopathology, genomics, demographics, and treatment information. It proposes ICFNet, a cross-modal fusion network that uses three encoders, a residual orthogonal decomposition module, and a unification fusion module to merge multi-modal features for improved accuracy, along with a balanced negative log-likelihood loss to ensure fair training. Experimental results on five public TCGA datasets (BLCA, BRCA, GBMLGG, LUAD, UCEC) demonstrate state-of-the-art performance, underscoring the method's robustness across cancer types. The authors provide open-source code, highlighting potential for clinical decision support and precision medicine.

Abstract

Survival prediction is a crucial task in the medical field and is essential for optimizing treatment options and resource allocation. However, current methods often rely on limited data modalities, resulting in suboptimal performance. In this paper, we propose an Integrated Cross-modal Fusion Network (ICFNet) that integrates histopathology whole slide images, genomic expression profiles, patient demographics, and treatment protocols. Specifically, three types of encoders, a residual orthogonal decomposition module and a unification fusion module are employed to merge multi-modal features to enhance prediction accuracy. Additionally, a balanced negative log-likelihood loss function is designed to ensure fair training across different patients. Extensive experiments demonstrate that our ICFNet outperforms state-of-the-art algorithms on five public TCGA datasets, including BLCA, BRCA, GBMLGG, LUAD, and UCEC, and shows its potential to support clinical decision-making and advance precision medicine. The codes are available at: https://github.com/binging512/ICFNet.