A Transformer-based Diffusion Probabilistic Model for Heart Rate and Blood Pressure Forecasting in Intensive Care Unit

TL;DR

This paper tackles the challenge of forecasting ICU vital signs under sparse, irregular data conditions. It introduces TDSTF, a Transformer-based Diffusion Probabilistic Model that uses a triplet representation to model sparse time series and learns $p_\theta(\mathbf{x}_0^p|\mathbf{x}_0^{co})$ to approximate the forward distribution $q(\mathbf{x}_0^{ta}|\mathbf{x}_0^{co})$. In extensive experiments on the MIMIC-III dataset, TDSTF outperforms state-of-the-art baselines in SACRPS and MSE by $18.9\%$ and $34.3\%$, respectively, while being over $17\times$ faster in inference than the leading diffusion model, CSDI. The approach demonstrates robust handling of sparse data and the ability to capture complex temporal dependencies across multiple vital signs, highlighting its potential for real-time ICU monitoring and decision support. Future work includes expanding to additional vitals and reducing false alarms to further support clinicians.

Abstract

Background and Objective: Vital sign monitoring in the Intensive Care Unit (ICU) is crucial for enabling prompt interventions for patients. This underscores the need for an accurate predictive system. Therefore, this study proposes a novel deep learning approach for forecasting Heart Rate (HR), Systolic Blood Pressure (SBP), and Diastolic Blood Pressure (DBP) in the ICU. Methods: We extracted $24,886$ ICU stays from the MIMIC-III database which contains data from over $46$ thousand patients, to train and test the model. The model proposed in this study, Transformer-based Diffusion Probabilistic Model for Sparse Time Series Forecasting (TDSTF), merges Transformer and diffusion models to forecast vital signs. The TDSTF model showed state-of-the-art performance in predicting vital signs in the ICU, outperforming other models' ability to predict distributions of vital signs and being more computationally efficient. The code is available at https://github.com/PingChang818/TDSTF. Results: The results of the study showed that TDSTF achieved a Standardized Average Continuous Ranked Probability Score (SACRPS) of $0.4438$ and a Mean Squared Error (MSE) of $0.4168$, an improvement of $18.9\%$ and $34.3\%$ over the best baseline model, respectively. The inference speed of TDSTF is more than $17$ times faster than the best baseline model. Conclusion: TDSTF is an effective and efficient solution for forecasting vital signs in the ICU, and it shows a significant improvement compared to other models in the field.

Figures (10)

• Figure 1: Diagram of the diffusion processes in our forecasting model. The blue curves indicate the history events as the conditional data $\mathbf{x}_0^{co}$. The red curves symbolize the noisy target $\mathbf{x}_t^{ta}$ at time step $t$ in the forward trajectory or the intermediate result $\mathbf{x}_t^p$ during prediction. The target is represented by $\mathbf{x}_0^{ta}$. The gaps among the curves symbolize the missing values in the sparse data. The forward trajectory $q$ adds noise of increasing levels to $\mathbf{x}_0^{ta}$. The reverse trajectory $p_\theta$ then removes the noise from the pure noise $\mathbf{x}_T^{p}$ to generate samples.
• Figure 2: The training procedure of our forecasting model minimizes the Mean Squared Error (MSE) between the noise prediction $\epsilon _\theta(x_t^{ta},t|x_0^{co})$ and $\epsilon$ as the loss function. The implementation of $\epsilon _\theta$ in the dashed box will be expanded and explained in detail later.
• Figure 3: An illustration of converting a sparse matrix to triplet representation. In this example, a patient received $200$ milligram (mg) of losartan at the second minute. A Heart Rate (HR) of $90$ beats per minute (bpm) and a Systolic Blood Pressure (SBP) of $130$ millimeters of mercury (mmHg) are recorded in the third and fifth minute, respectively. Assuming all other data points in the matrix are missing, and the $3$ valid event records $\textbf{a}$, $\textbf{b}$, and $\textbf{c}$ line up in an array of triplets. The size of the input triplet arrays is preset, and it can be larger than the number of valid triplets. The mask value of $0$ in $\textbf{d}$ signifies the invalidity of this triplet, meaning the invalidity of its other $3$ elements.
• Figure 4: The architecture of the $\epsilon_\theta$ network, designed for predicting $\epsilon$ at the current step.
• Figure 5: Illustration of the data preprocessing steps involved in this study. $n$ denotes the number of ICU stays.