Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Pulse-Signal Magnification for remote Heart Rate Estimation in Compressed Videos

Joaquim Comas, Adria Ruiz, Federico Sukno

TL;DR

A novel approach to address the impact of video compression on rPPG estimation is presented, which leverages a pulse-signal magnification transformation to adapt compressed videos to an uncompressed data domain in which the rPPG signal is magnified.

Abstract

Recent advancements in data-driven approaches for remote photoplethysmography (rPPG) have significantly improved the accuracy of remote heart rate estimation. However, the performance of such approaches worsens considerably under video compression, which is nevertheless necessary to store and transmit video data efficiently. In this paper, we present a novel approach to address the impact of video compression on rPPG estimation, which leverages a pulse-signal magnification transformation to adapt compressed videos to an uncompressed data domain in which the rPPG signal is magnified. We validate the effectiveness of our model by exhaustive evaluations on two publicly available datasets, UCLA-rPPG and UBFC-rPPG, employing both intra- and cross-database performance at several compression rates. Additionally, we assess the robustness of our approach on two additional highly compressed and widely-used datasets, MAHNOB-HCI and COHFACE, which reveal outstanding heart rate estimation results.

Deep Pulse-Signal Magnification for remote Heart Rate Estimation in Compressed Videos

TL;DR

A novel approach to address the impact of video compression on rPPG estimation is presented, which leverages a pulse-signal magnification transformation to adapt compressed videos to an uncompressed data domain in which the rPPG signal is magnified.

Abstract

Recent advancements in data-driven approaches for remote photoplethysmography (rPPG) have significantly improved the accuracy of remote heart rate estimation. However, the performance of such approaches worsens considerably under video compression, which is nevertheless necessary to store and transmit video data efficiently. In this paper, we present a novel approach to address the impact of video compression on rPPG estimation, which leverages a pulse-signal magnification transformation to adapt compressed videos to an uncompressed data domain in which the rPPG signal is magnified. We validate the effectiveness of our model by exhaustive evaluations on two publicly available datasets, UCLA-rPPG and UBFC-rPPG, employing both intra- and cross-database performance at several compression rates. Additionally, we assess the robustness of our approach on two additional highly compressed and widely-used datasets, MAHNOB-HCI and COHFACE, which reveal outstanding heart rate estimation results.
Paper Structure (30 sections, 7 equations, 4 figures, 6 tables)

This paper contains 30 sections, 7 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Contribution
  3. Related work
  4. Camera-based Physiological measurement
  5. Video-compression in existing rPPG datasets
  6. Influence of video compression on rPPG estimation
  7. rPPG solutions for compressed facial videos
  8. Methodology
  9. Problem formulation
  10. Networks Architecture
  11. Two-stage Training Framework
  12. Stage I
  13. Stage II
  14. Loss function
  15. Constrained Pulse-Signal Magnification space
  16. ...and 15 more sections

Figures (4)

  • Figure 1: Overall structure of our proposed model, which has two stages for rPPG recovery under compression. Firstly, the TDM model is trained on uncompressed data. Then, the Pulse-Signal Magnification network is trained on compressed data with fixed TDM model parameters.
  • Figure 2: Training procedure evaluation for rPPG recovery in intra-dataset and cross-dataset evaluation.
  • Figure 3: Visualization of the learned video transformation for a sample compressed at CRF 15 from the UCLA-rPPG dataset. The top section displays the generated video, $m_\psi(\mathbf{C}^n)$ between frames 80 and 210. In the middle, we see the predicted rPPG and ground-truth signals. The bottom section shows the number of pixels over time for green and magenta colors, which are the dominant ones after the transformation and capture the blood pulse effect (see also the https://youtu.be/bIv4itnT2mE).
  • Figure 4: Comparison of rPPG estimations between our baseline (green) and proposed model (blue) in the COHFACE dataset.