Advancing Generalizable Remote Physiological Measurement through the Integration of Explicit and Implicit Prior Knowledge

TL;DR

This work tackles limited cross-dataset generalization in remote photoplethysmography by introducing Greip, a framework that fuses explicit priors (camera, motion, illumination, skin tone) with an implicit prior realized through a dual-branch network that disentangles physiology from noise. It leverages STMap-based augmented inputs, a feature continuity constraint, and an orthogonality constraint to cultivate a continuous, physiology-aligned rPPG feature space while suppressing noise interference. The approach yields substantial gains in HR and HRV estimation across multiple source and target datasets, enables RGB to NIR generalization, and proves effective under self-supervised and semi-supervised regimes, as well as in a 3D mask PAD setting. These results highlight Greip as a robust, generalizable solution for real-world, cross-domain rPPG applications with broad practical impact.

Abstract

Remote photoplethysmography (rPPG) is a promising technology that captures physiological signals from face videos, with potential applications in medical health, emotional computing, and biosecurity recognition. The demand for rPPG tasks has expanded from demonstrating good performance on intra-dataset testing to cross-dataset testing (i.e., domain generalization). However, most existing methods have overlooked the prior knowledge of rPPG, resulting in poor generalization ability. In this paper, we propose a novel framework that simultaneously utilizes explicit and implicit prior knowledge in the rPPG task. Specifically, we systematically analyze the causes of noise sources (e.g., different camera, lighting, skin types, and movement) across different domains and incorporate these prior knowledge into the network. Additionally, we leverage a two-branch network to disentangle the physiological feature distribution from noises through implicit label correlation. Our extensive experiments demonstrate that the proposed method not only outperforms state-of-the-art methods on RGB cross-dataset evaluation but also generalizes well from RGB datasets to NIR datasets. The code is available at https://github.com/keke-nice/Greip.

Figures (6)

• Figure 1: The framework of Greip to utilize the explicit and implicit prior knowledge. Firstly, we incorporate explicit priors into the network in a unified augmentation way. Subsequently, we utilize the continuous implicit prior of rPPG labels to impose constraints on the rPPG features and noise within the network, which effectively transforms the network from a chaotic feature space into a distinguishable and continuous one.
• Figure 2: An overview of the proposed method. The above part shows the source and composition of the explicit prior in the collection process of the rPPG datasets. The following part shows the architecture of the entire two-flow network and how to constrain the rPPG feature and the implicit noise distribution, and finally inject the noise into the rPPG feature.
• Figure 3: Visualization of explicit priors. We visualized the five explicit augments mentioned in the Section \ref{['sec:Explicit']}, with different colors representing the three color channels: red, green, and blue. All the augmentation strategies are implemented on STMap.
• Figure 4: Impacts of the hyperparameter (a) K and (b) t of the proposed method.
• Figure 5: Visualization of the rPPG feature. The heart rate value represented by the feature increases as the color lightens.