Phase-shifted remote photoplethysmography for estimating heart rate and blood pressure from facial video

TL;DR

This work tackles non-contact estimation of $HR$ and $SBP$/$DBP$ from facial video by introducing phase-shifted $rPPG$ signals. It presents a two-stage framework with DRP-Net for extracting phase-shifted acral and facial $rPPG$ and BBP-Net for BP estimation from phase-discrepancy features, aided by frame-interpolation augmentation and a bounded sigmoid BP head. The approach achieves state-of-the-art results on MMSE-HR and V4V datasets, with detailed ablations and cross skin-tone analyses highlighting both strengths and limitations. The study advances contactless cardiovascular monitoring while noting the need for facial-site $PPG$ ground truth data to further validate and generalize the method across diverse populations.

Abstract

Human health can be critically affected by cardiovascular diseases, such as hypertension, arrhythmias, and stroke. Heart rate and blood pressure are important biometric information for the monitoring of cardiovascular system and early diagnosis of cardiovascular diseases. Existing methods for estimating the heart rate are based on electrocardiography and photoplethyomography, which require contacting the sensor to the skin surface. Moreover, catheter and cuff-based methods for measuring blood pressure cause inconvenience and have limited applicability. Therefore, in this thesis, we propose a vision-based method for estimating the heart rate and blood pressure. This thesis proposes a 2-stage deep learning framework consisting of a dual remote photoplethysmography network (DRP-Net) and bounded blood pressure network (BBP-Net). In the first stage, DRP-Net infers remote photoplethysmography (rPPG) signals for the acral and facial regions, and these phase-shifted rPPG signals are utilized to estimate the heart rate. In the second stage, BBP-Net integrates temporal features and analyzes phase discrepancy between the acral and facial rPPG signals to estimate SBP and DBP values. To improve the accuracy of estimating the heart rate, we employed a data augmentation method based on a frame interpolation model. Moreover, we designed BBP-Net to infer blood pressure within a predefined range by incorporating a scaled sigmoid function. Our method resulted in estimating the heart rate with the mean absolute error (MAE) of 1.78 BPM, reducing the MAE by 34.31 % compared to the recent method, on the MMSE-HR dataset. The MAE for estimating the systolic blood pressure (SBP) and diastolic blood pressure (DBP) were 10.19 mmHg and 7.09 mmHg. On the V4V dataset, the MAE for the heart rate, SBP, and DBP were 3.83 BPM, 13.64 mmHg, and 9.4 mmHg, respectively.

Figures (10)

• Figure 1: Overview of the training pipeline of the proposed method. Red and blue texts indicate ground truth and predicted physiological information, respectively. The green, red, and blue arrows represent the preprocessing of input videos, the generation of ground truth, and the post-processing of model outputs, respectively.
• Figure 2: Data augmentation process of bradycardia and tachycardia samples.
• Figure 3: Architecture of DRP-Net.
• Figure 4: Architecture of BBP-Net.
• Figure 5: Visualization of rPPG signals (left) and their PSD (right). The upper left corner of each rPPG signal displays the reference heart rate.