BioFusionNet: Deep Learning-Based Survival Risk Stratification in ER+ Breast Cancer Through Multifeature and Multimodal Data Fusion

TL;DR

ER+ breast cancer prognosis remains challenging due to heterogeneity across imaging, genomics, and clinical data. BioFusionNet tackles this with a multimodal pipeline that fuses self-supervised histopathology features from DINO and MoCoV3 with gene expression and clinical variables via a variational autoencoder, co-dual-cross-attention, and late fusion, optimized by a weighted Cox loss. The model achieves a mean C-index of $0.77$ and a time-dependent AUC of $0.84$, with significant hazard stratification (OS HR ≈ $2.99$ univariate, $2.91$ multivariate) and interpretability through attention maps and SHAP analyses. This approach demonstrates the value of comprehensive data integration for accurate prognosis in ER+ breast cancer and offers a foundation for clinical translation, though it carries substantial computational cost and requires broader validation across datasets and cancer types.

Abstract

Breast cancer is a significant health concern affecting millions of women worldwide. Accurate survival risk stratification plays a crucial role in guiding personalised treatment decisions and improving patient outcomes. Here we present BioFusionNet, a deep learning framework that fuses image-derived features with genetic and clinical data to obtain a holistic profile and achieve survival risk stratification of ER+ breast cancer patients. We employ multiple self-supervised feature extractors (DINO and MoCoV3) pretrained on histopathological patches to capture detailed image features. These features are then fused by a variational autoencoder and fed to a self-attention network generating patient-level features. A co-dual-cross-attention mechanism combines the histopathological features with genetic data, enabling the model to capture the interplay between them. Additionally, clinical data is incorporated using a feed-forward network, further enhancing predictive performance and achieving comprehensive multimodal feature integration. Furthermore, we introduce a weighted Cox loss function, specifically designed to handle imbalanced survival data, which is a common challenge. Our model achieves a mean concordance index of 0.77 and a time-dependent area under the curve of 0.84, outperforming state-of-the-art methods. It predicts risk (high versus low) with prognostic significance for overall survival in univariate analysis (HR=2.99, 95% CI: 1.88--4.78, p<0.005), and maintains independent significance in multivariate analysis incorporating standard clinicopathological variables (HR=2.91, 95\% CI: 1.80--4.68, p<0.005).

Figures (7)

• Figure 1: Illustration of the clinical management pathway in treating breast cancer patients. This example concerns a postmenopausal patient who has been diagnosed with breast cancer, specifically invasive ductal carcinoma (IDC). The process begins with an initial diagnosis through clinical examination, imaging and core biopsy. Following this, surgery is performed to completely excise the tumour, and postoperative tumour histopathological classification is performed to assess key factors including tumour size (e.g. T2), grade (e.g. G2), hormone receptor status (e.g. ER+), HER2 status (e.g. HER2-), lymph node status (e.g. LN-), and proliferation index (e.g. Ki67 20%). Subsequent treatments may include hormone therapy and radiotherapy. Additional molecular tests, like genetic testing, are utilised to determine specific cancer molecular subtypes and further assess risk of recurrence. The proposed final step in this pathway is the application of our BioFusionNet model. This model combines tumour characteristics, pathology and genetic testing to determine high and low risk patients, thereby guiding personalised treatment decisions and efficiently preventing both under-treatment and over-treatment. For example, low-risk patients might undergo lumpectomy with hormone therapy and radiotherapy, whereas high-risk patients are advised to have chemotherapy in addition to these treatments. Whilst this pathway mirrors current clinical practice, our study streamlines the integration of all available critical data to derive an automated single risk prediction score.
• Figure 2: BioFusionNet Stage 1: The proposed model integrates self-supervised image feature extraction methods, namely DINO and MoCoV3, pretrained on three distinct datasets. Features are concatenated and fed to a Variational AutoEncoder (VAE). Subsequently, the latent space of the VAE is utilised to feed a self-attention network, which aggregates patch-level features into a comprehensive patient-level representation.
• Figure 3: BioFusionNet Stage 2: The proposed model fuses image embeddings generated from Stage 1 with genetic data through a co-dual-cross-attention mechanism. This fusion is subsequently combined with clinical data using a feed-forward network (FFN), leading to the generation of the final risk score output.
• Figure 4: Performance comparison of BioFusionNet and other methods using Kaplan-Meier survival curves.
• Figure 5: Visualisation of model-derived attention regions and associated risk types in Luminal A and Luminal B breast cancer patients. The figure presents raw histopathological image patches processed with BioFusionNet, which identifies areas of high and low attention, subsequently categorising patients into high and low risk.