VideoPulse: Neonatal heart rate and peripheral capillary oxygen saturation (SpO2) estimation from contact free video

TL;DR

VideoPulse, a neonatal dataset and an end to end pipeline that estimates neonatal heart rate and peripheral capillary oxygen saturation from facial video, indicates that short unaligned neonatal video segments can support accurate heart rate and SpO2 estimation, enabling low cost non invasive monitoring in neonatal intensive care.

Abstract

Remote photoplethysmography (rPPG) enables contact free monitoring of vital signs and is especially valuable for neonates, since conventional methods often require sustained skin contact with adhesive probes that can irritate fragile skin and increase infection control burden. We present VideoPulse, a neonatal dataset and an end to end pipeline that estimates neonatal heart rate and peripheral capillary oxygen saturation (SpO2) from facial video. VideoPulse contains 157 recordings totaling 2.6 hours from 52 neonates with diverse face orientations. Our pipeline performs face alignment and artifact aware supervision using denoised pulse oximeter signals, then applies 3D CNN backbones for heart rate and SpO2 regression with label distribution smoothing and weighted regression for SpO2. Predictions are produced in 2 second windows. On the NBHR neonatal dataset, we obtain heart rate MAE 2.97 bpm using 2 second windows (2.80 bpm at 6 second windows) and SpO2 MAE 1.69 percent. Under cross dataset evaluation, the NBHR trained heart rate model attains 5.34 bpm MAE on VideoPulse, and fine tuning an NBHR pretrained SpO2 model on VideoPulse yields MAE 1.68 percent. These results indicate that short unaligned neonatal video segments can support accurate heart rate and SpO2 estimation, enabling low cost non invasive monitoring in neonatal intensive care.

Figures (6)

• Figure 1: System architecture. The proposed HR and SpO2 prediction pipeline is shown for both adults (only SpO2), and neonates (both SpO2 and HR). The model was trained the NBHR dataset for neonates and further fine-tuned on the VideoPulse dataset.
• Figure 2: Ground truth PPG denoising.A: Raw PPG signals from the oximeter containing motion artifacts. These are first classified by an SVM to identify noisy regions. B: The reconstructed signal after the noisy segments were reconstructed using a GAN. C: The reconstructed signal is further filtered to recover clean PPG segments for reliable rPPG analysis. The red shaded regions are removed since they are too noisy.
• Figure 3: Schematic of the VideoPulse data collection setup. A webcam mounted on a tripod records RGB video of the neonate in synchronization with a pulse oximeter measuring ground truth PPG data.
• Figure 4: Neonatal Heart Rate estimation results. A: Evaluation on the test split of NBHR dataset. B: Evaluation on the test split of VideoPulse dataset. Predictions align closely with the identity line and errors remain centered near zero.
• Figure 5: Neonatal SpO2 estimation on the NBHR dataset. The scatter plot of predicted vs. ground truth values is shown with the red dashed line indicating the ideal $y=x$ reference. The predicted values have a reasonable correspondence with the ground truth values.