Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Uncertain Demand Prediction in Hospitals Using Simple and Advanced Machine Learning

Annie Hu, Samuel Stockman, Xun Wu, Richard Wood, Bangdong Zhi, Oliver Y. Chén

TL;DR

This work tackles the challenge of uncertain, time-varying patient care demand in hospitals by comparing a time-varying linear model and an LSTM for hourly-to-short-horizon forecasts using Rambam Medical Center data. The time-varying linear model employs a Kalman filter–based state-space framework for online adaptation and interpretability, while the LSTM captures nonlinear temporal dynamics and weekly seasonal effects, potentially enhanced by exogenous features like temperature. Across naive baselines and competing methods (RVAR and TBATS), both proposed approaches demonstrate strong predictive capability, with the LSTM variant offering the lowest prediction errors and the linear model delivering robust performance with greater transparency. The study demonstrates that machine-learning–driven forecast tools can achieve practical accuracy (approximately 4 patients) up to 3–7 days ahead, informing hospital resource planning and decision-making, and suggests avenues for online learning and ensemble methods to further improve performance.

Abstract

Early and timely prediction of patient care demand not only affects effective resource allocation but also influences clinical decision-making as well as patient experience. Accurately predicting patient care demand, however, is a ubiquitous challenge for hospitals across the world due, in part, to the demand's time-varying temporal variability, and, in part, to the difficulty in modelling trends in advance. To address this issue, here, we develop two methods, a relatively simple time-vary linear model, and a more advanced neural network model. The former forecasts patient arrivals hourly over a week based on factors such as day of the week and previous 7-day arrival patterns. The latter leverages a long short-term memory (LSTM) model, capturing non-linear relationships between past data and a three-day forecasting window. We evaluate the predictive capabilities of the two proposed approaches compared to two naïve approaches - a reduced-rank vector autoregressive (VAR) model and the TBATS model. Using patient care demand data from Rambam Medical Center in Israel, our results show that both proposed models effectively capture hourly variations of patient demand. Additionally, the linear model is more explainable thanks to its simple architecture, whereas, by accurately modelling weekly seasonal trends, the LSTM model delivers lower prediction errors. Taken together, our explorations suggest the utility of machine learning in predicting time-varying patient care demand; additionally, it is possible to predict patient care demand with good accuracy (around 4 patients) three days or a week in advance using machine learning.

Enhancing Uncertain Demand Prediction in Hospitals Using Simple and Advanced Machine Learning

TL;DR

This work tackles the challenge of uncertain, time-varying patient care demand in hospitals by comparing a time-varying linear model and an LSTM for hourly-to-short-horizon forecasts using Rambam Medical Center data. The time-varying linear model employs a Kalman filter–based state-space framework for online adaptation and interpretability, while the LSTM captures nonlinear temporal dynamics and weekly seasonal effects, potentially enhanced by exogenous features like temperature. Across naive baselines and competing methods (RVAR and TBATS), both proposed approaches demonstrate strong predictive capability, with the LSTM variant offering the lowest prediction errors and the linear model delivering robust performance with greater transparency. The study demonstrates that machine-learning–driven forecast tools can achieve practical accuracy (approximately 4 patients) up to 3–7 days ahead, informing hospital resource planning and decision-making, and suggests avenues for online learning and ensemble methods to further improve performance.

Abstract

Early and timely prediction of patient care demand not only affects effective resource allocation but also influences clinical decision-making as well as patient experience. Accurately predicting patient care demand, however, is a ubiquitous challenge for hospitals across the world due, in part, to the demand's time-varying temporal variability, and, in part, to the difficulty in modelling trends in advance. To address this issue, here, we develop two methods, a relatively simple time-vary linear model, and a more advanced neural network model. The former forecasts patient arrivals hourly over a week based on factors such as day of the week and previous 7-day arrival patterns. The latter leverages a long short-term memory (LSTM) model, capturing non-linear relationships between past data and a three-day forecasting window. We evaluate the predictive capabilities of the two proposed approaches compared to two naïve approaches - a reduced-rank vector autoregressive (VAR) model and the TBATS model. Using patient care demand data from Rambam Medical Center in Israel, our results show that both proposed models effectively capture hourly variations of patient demand. Additionally, the linear model is more explainable thanks to its simple architecture, whereas, by accurately modelling weekly seasonal trends, the LSTM model delivers lower prediction errors. Taken together, our explorations suggest the utility of machine learning in predicting time-varying patient care demand; additionally, it is possible to predict patient care demand with good accuracy (around 4 patients) three days or a week in advance using machine learning.
Paper Structure (14 sections, 2 theorems, 25 equations, 4 figures, 2 tables)

This paper contains 14 sections, 2 theorems, 25 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Exploratory Data Analysis
  4. Forecasting Model
  5. Naive Approach
  6. Reduced-rank Vector Autoregression Model (RVAR)
  7. Time-varying Linear Model
  8. TBATS Model
  9. Long Short-Term Memory
  10. Results
  11. Conclusion
  12. Proof of Lemmas and Propositions
  13. Proof of Theorem \ref{['marginal']}
  14. Proof of Proposition \ref{['prop1']}

Key Result

Theorem 1

Let $X$ and $M$ be random variables with distributions: $p(X|M = m)=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{1}{2}\frac{(X-m)^2}{\sigma^2}}$, and $p(M)=\frac{1}{s\sqrt{2\pi}}e^{-\frac{1}{2}\frac{(M-\theta)^2}{s^2}}$, then the marginal distribution $p(X)=\mathcal{N}(X;\theta,s^2+\sigma^2)$.

Figures (4)

  • Figure 1: (a) Boxplot of the number of patients arriving at each hour in the day to the Rambam Medical Center.(b) The number of patients arriving on each day of the week.
  • Figure 2: Boxplot of the number of patients arriving at the Rambam Medical Center at each hour of the week beginning 00:00 Monday.
  • Figure 3: (a) Boxplot of the number of patients arriving at the Rambam Medical Center per hour for each month of the year. (b) The number of patients that arrive on each week of the year.
  • Figure 4: Forecasts of patients arriving at the Rambam Medical Center by both the LSTM and linear model. Blue forecasts from the LSTM model and orange from the linear model are compared with the true observed points in green.

Theorems & Definitions (2)

  • Theorem 1
  • Proposition 1