Towards accurate and reliable ICU outcome prediction: a multimodal learning framework based on belief function theory using structured EHRs and free-text notes

TL;DR

This work introduces a multimodal ICU outcome prediction framework based on belief function theory to fuse structured EHR data with free-text notes. By mapping deep features from each modality into belief evidence via an evidential neural network and fusing them with Dempster’s rule, the approach captures uncertainty and conflicts across data sources. On the MIMIC-III dataset, it achieves higher predictive accuracy and markedly improved reliability (lower Brier score and NLL) than baselines, while reducing false positives to aid ICU resource allocation. The framework leverages encoders such as FT-Transformer for structured data and Clinical BERT for notes, and it demonstrates potential for extending to other clinical tasks with additional modalities and multiclass or regression targets.

Abstract

Accurate Intensive Care Unit (ICU) outcome prediction is critical for improving patient treatment quality and ICU resource allocation. Existing research mainly focuses on structured data, e.g. demographics and vital signs, and lacks effective frameworks to integrate clinical notes from heterogeneous electronic health records (EHRs). This study aims to explore a multimodal framework based on belief function theory that can effectively fuse heterogeneous structured EHRs and free-text notes for accurate and reliable ICU outcome prediction. The fusion strategy accounts for prediction uncertainty within each modality and conflicts between multimodal data. The experiments on MIMIC-III dataset show that our framework provides more accurate and reliable predictions than existing approaches. Specifically, it outperformed the best baseline by 1.05%/1.02% in BACC, 9.74%/6.04% in F1 score, 1.28%/0.9% in AUROC, and 6.21%/2.68% in AUPRC for predicting mortality and PLOS, respectively. Additionally, it improved the reliability of the predictions with a 26.8%/15.1% reduction in the Brier score and a 25.0%/13.3% reduction in negative log-likelihood. By effectively reducing false positives, the model can aid in better allocation of medical resources in the ICU. Furthermore, the proposed method is very versatile and can be extended to analyzing multimodal EHRs for other clinical tasks. The code implementation is available on https://github.com/yuchengruan/evid_multimodal_ehr.

Figures (11)

• Figure 1: The illustration of ENN
• Figure 2: The overview of our proposed framework. EM: evidence mapping and EF: evidence fusion
• Figure 3: The illustration of different fusion settings.
• Figure 4: The evaluation of our framework using Clinical BERT as text encoder on different fusion settings for mortality prediction: (1) modalities, (2) data types, (3) data sources.
• Figure 5: The evaluation of our framework using Clinical BERT as text encoder on different fusion settings for PLOS prediction: (1) modalities, (2) data types, (3) data sources.