DT-ICU: Towards Explainable Digital Twins for ICU Patient Monitoring via Multi-Modal and Multi-Task Iterative Inference

TL;DR

DT-ICU presents a multimodal digital twin for continuous ICU monitoring that updates mortality risk and future physiological trajectories as heterogeneous data accumulate. The model uses variable-length time-series and static patient information within a multitask framework enhanced by modality-specific encoders, a modality transformer, a causal temporal transformer, and cross-attention with static context. On MIMIC-IV, DT-ICU outperforms literature baselines and standard backbones, remains robust to input length, and reveals a clinically meaningful hierarchy of modality contributions via LOMO/LTMO analyses. The work demonstrates early discriminative ability (within 12–24 hours) and improved high-risk patient ranking with longer observation windows, while enabling forward simulation for what-if analyses. The findings support DT-ICU as a practical, interpretable digital twin for continuous ICU monitoring, with code and weights publicly available for reproducibility.

Abstract

We introduce DT-ICU, a multimodal digital twin framework for continuous risk estimation in intensive care. DT-ICU integrates variable-length clinical time series with static patient information in a unified multitask architecture, enabling predictions to be updated as new observations accumulate over the ICU stay. We evaluate DT-ICU on the large, publicly available MIMIC-IV dataset, where it consistently outperforms established baseline models under different evaluation settings. Our test-length analysis shows that meaningful discrimination is achieved shortly after admission, while longer observation windows further improve the ranking of high-risk patients in highly imbalanced cohorts. To examine how the model leverages heterogeneous data sources, we perform systematic modality ablations, revealing that the model learnt a reasonable structured reliance on interventions, physiological response observations, and contextual information. These analyses provide interpretable insights into how multimodal signals are combined and how trade-offs between sensitivity and precision emerge. Together, these results demonstrate that DT-ICU delivers accurate, temporally robust, and interpretable predictions, supporting its potential as a practical digital twin framework for continuous patient monitoring in critical care. The source code and trained model weights for DT-ICU are publicly available at https://github.com/GUO-W/DT-ICU-release.

Figures (5)

• Figure 1: Effect of test length on predictive performance. The left panel reports AUROC, Accuracy, and AUPRC under different test lengths (hours), while the right panel reports Precision and Recall, highlighting the precision--recall trade-off as the observation window increases.
• Figure 2: Leave-One-Modality-Out (LOMO) ablation analysis showing performance degradation when individual modalities are removed during inference. Each bar represents the change in performance ($\Delta$) relative to the baseline model using all modalities, where negative values indicate performance drops.
• Figure 3: Leave-Two-Modalities-Out (LTMO) ablation heatmaps showing performance changes when pairs of modalities are simultaneously removed during inference. Each cell $(i,j)$ represents the change $\Delta$ when both modality $i$ and modality $j$ are zeroed out. Blue intensity indicates severity of performance degradation (darker = worse), white indicates little or no change, and red indicates improvement (rare). Diagonal cells are masked. Panels (a--e) report $\Delta$AUROC, $\Delta$AUPRC, $\Delta$Accuracy, $\Delta$Precision, and $\Delta$Recall changes.
• Figure 4: Pairwise metric relationships. Each point represents one two-modality zero-out combination (28 pairs across 8 modalities), with the baseline (all modalities) marked for reference. (a)) AUROC and AUPRC show a strong positive correlation (Pearson $\rho=0.975$, $p<0.0001$). (b) Precision and recall have a systematic trade-off pattern (Pearson $\rho=-0.425$, $p=0.024$), with distinct clusters corresponding to different modality-dependent failure modes.
• Figure 5: Pipeline of our proposed DT-ICU model. Multimodal ICU data up to time $t$ are encoded into modality-specific embeddings and processed by a modality transformer followed by a causal temporal transformer. Static patient attributes (demographics and diagnoses) are fused with the temporal representation via cross-attention. PE indicates positional encoding, and FC indicates fully connected layers. The model outputs both a mortality prediction and next-step physiological forecasts at time $t{+}1$ through multitask heads. When new ICU observations become available, predictions could be updated using real measurements. When future observations are not yet available, the model rolls out forward in time by feeding its own regression outputs back as inputs, enabling iterative future inference.