Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Camera-Based HRV Prediction for Remote Learning Environments

Kegang Wang, Yantao Wei, Jiankai Tang, Yuntao Wang, Mingwen Tong, Jie Gao, Yujian Ma, Zhongjin Zhao

TL;DR

Using the RLAP dataset, a new model called Seq-rPPG is trained, it is a model based on one-dimensional convolution, and experimental results reveal that this structure is more suitable for handling HRV tasks, which outperformed all other baselines in HRV performance and also demonstrated significant advantages in computational efficiency.

Abstract

In recent years, due to the widespread use of internet videos, remote photoplethysmography (rPPG) has gained more and more attention in the fields of affective computing. Restoring blood volume pulse (BVP) signals from facial videos is a challenging task that involves a series of preprocessing, image algorithms, and postprocessing to restore waveforms. Not only is the heart rate metric utilized for affective computing, but the heart rate variability (HRV) metric is even more significant. The challenge in obtaining HRV indices through rPPG lies in the necessity for algorithms to precisely predict the BVP peak positions. In this paper, we collected the Remote Learning Affect and Physiology (RLAP) dataset, which includes over 32 hours of highly synchronized video and labels from 58 subjects. This is a public dataset whose BVP labels have been meticulously designed to better suit the training of HRV models. Using the RLAP dataset, we trained a new model called Seq-rPPG, it is a model based on one-dimensional convolution, and experimental results reveal that this structure is more suitable for handling HRV tasks, which outperformed all other baselines in HRV performance and also demonstrated significant advantages in computational efficiency.

Camera-Based HRV Prediction for Remote Learning Environments

TL;DR

Using the RLAP dataset, a new model called Seq-rPPG is trained, it is a model based on one-dimensional convolution, and experimental results reveal that this structure is more suitable for handling HRV tasks, which outperformed all other baselines in HRV performance and also demonstrated significant advantages in computational efficiency.

Abstract

In recent years, due to the widespread use of internet videos, remote photoplethysmography (rPPG) has gained more and more attention in the fields of affective computing. Restoring blood volume pulse (BVP) signals from facial videos is a challenging task that involves a series of preprocessing, image algorithms, and postprocessing to restore waveforms. Not only is the heart rate metric utilized for affective computing, but the heart rate variability (HRV) metric is even more significant. The challenge in obtaining HRV indices through rPPG lies in the necessity for algorithms to precisely predict the BVP peak positions. In this paper, we collected the Remote Learning Affect and Physiology (RLAP) dataset, which includes over 32 hours of highly synchronized video and labels from 58 subjects. This is a public dataset whose BVP labels have been meticulously designed to better suit the training of HRV models. Using the RLAP dataset, we trained a new model called Seq-rPPG, it is a model based on one-dimensional convolution, and experimental results reveal that this structure is more suitable for handling HRV tasks, which outperformed all other baselines in HRV performance and also demonstrated significant advantages in computational efficiency.
Paper Structure (14 sections, 4 figures, 6 tables)

This paper contains 14 sections, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Dataset
  3. Program Coding
  4. Dataset Collection
  5. Algorithm
  6. Experiment and Results
  7. Datasets and Metrics
  8. Intra-dataset Testing
  9. Cross-dataset Testing
  10. Computational Overhead
  11. Discussion
  12. RLAP Dataset
  13. Seq-rPPG Method
  14. Conclusion

Figures (4)

  • Figure 1: Overview of the RLAP dataset. The RLAP dataset comprises 58 student samples, encompassing various scenarios, emotions, and levels of learning engagement. While completing these tasks, participants' pulse signals were synchronously recorded with a pulse oximeter.
  • Figure 2: Seq-rPPG network, it consists of an encoder and a decoder. The encoder is a single-layer 1x1 convolution, while the decoder comprises four layers of alternating time-domain and frequency-domain convolutions.
  • Figure 3: The relationship between the time window size of Seq-rPPG and the SDNN error.
  • Figure 4: The BVP waveforms output by four models during head movements, where the solid lines indicate the size of the time window.