Continuous Patient Monitoring with AI: Real-Time Analysis of Video in Hospital Care Settings

TL;DR

This work tackles the challenge of continuous, real-time patient monitoring in hospitals by deploying an AI-driven video-analysis platform across 11 hospitals to track high-risk patients. It combines object detection, patient-role classification, and motion estimation to derive logical and trend predictions, with a publicly released anonymized dataset of over 1,000 patient-days. The results show strong frame-level performance (e.g., F1 up to 0.98 for patient-role and 0.92 overall for objects) and robust trend alignment with ground-truth observations, even under privacy-preserving blur and camera-position variability. The study highlights practical implications for safety, staffing, and care planning, while outlining privacy, standardization, and scalability challenges and providing a benchmark for future AI-driven patient monitoring systems.

Abstract

This study introduces an AI-driven platform for continuous and passive patient monitoring in hospital settings, developed by LookDeep Health. Leveraging advanced computer vision, the platform provides real-time insights into patient behavior and interactions through video analysis, securely storing inference results in the cloud for retrospective evaluation. The dataset, compiled in collaboration with 11 hospital partners, encompasses over 300 high-risk fall patients and over 1,000 days of inference, enabling applications such as fall detection and safety monitoring for vulnerable patient populations. To foster innovation and reproducibility, an anonymized subset of this dataset is publicly available. The AI system detects key components in hospital rooms, including individual presence and role, furniture location, motion magnitude, and boundary crossings. Performance evaluation demonstrates strong accuracy in object detection (macro F1-score = 0.92) and patient-role classification (F1-score = 0.98), as well as reliable trend analysis for the "patient alone" metric (mean logistic regression accuracy = 0.82 \pm 0.15). These capabilities enable automated detection of patient isolation, wandering, or unsupervised movement-key indicators for fall risk and other adverse events. This work establishes benchmarks for validating AI-driven patient monitoring systems, highlighting the platform's potential to enhance patient safety and care by providing continuous, data-driven insights into patient behavior and interactions.

Figures (9)

• Figure 1: Illustrative Workflow of the LookDeep Health AI-driven Patient Monitoring Platform. The system captures video from a hospital room using mounted cameras and processes each image through a series of computer vision modules. The output is presented as real-time insights for healthcare staff.
• Figure 2: Overview of Patient Demographics. (A) Observations subset, comprising 10 patients, 54 patient-days, and 3 hospitals. (B) Released dataset, comprising 387 patients, 1,466 patient-days, and 11 hospitals. Left: Pie charts showing distribution of hospitals by size. Hospitals are grouped by average daily census. Center: Heat maps showing patient age distribution by gender. Right: Box plots showing patient length of monitoring. Central line represents the median, box edges indicate the 25th and 75th percentiles, and whiskers extend to the most extreme data points within 1.5 times the interquartile range. The points represent outliers beyond this range. The released dataset shows a broader demographic and extended data duration compared to the observations subset.
• Figure 3: Dataset Overview and Timeline of Model Updates. Progression of data collection and model updates for the LookDeep Health monitoring system. Single-frame analysis data collection spans a two year period - a broad temporal range for training and validation of object detection and classification tasks. Observation logging data, used for trend validation, was collected over a one year period. The publicly released dataset includes data from a more recent six month period, representing over 1,000 collective patient days. Model updates are indicated by numbered points.
• Figure 4: Real-Time Object Detection, Motion Analysis, and Patient Status Monitoring. Top-left: Object detection with bounding boxes. Top-right: Segmentation map (red = safety zone, green = motion). Middle: "Alone" logical trend over time for every second within the hour. Bottom: "Alone" trend over a 24-hour period, aggregated for each hour. The black markers in the middle and bottom rows correspond to the time of the top row.
• Figure 5: Camera Setup and Example Frames. (A) LookDeep Video Unit (LVU), a 6” x 6” device, in various mounting configurations. (B) A 3x3 grid of representative frames captured by the system, showing a diversity of configurations. All images are intentionally blurred to maintain privacy. Each numbered frame provides a unique example that is found in Figure \ref{['fig:camera_meta']}.