Resolve Domain Conflicts for Generalizable Remote Physiological Measurement

TL;DR

The paper tackles domain conflicts in multi-dataset remote photoplethysmography (rPPG) by identifying label conflicts from phase delays and attribute conflicts from external noise. It introduces the DOmain-HArmonious framework (DOHA), combining Harmonius Phase Strategy (HPS) that yields a self-similarity representation to stabilize labels, and Harmonious Hyperplane Optimization (HHO) that uses Global Gradient Harmony (GGH) and Instance-wise Gradient Harmony (IGH) to harmonize gradients and reach a global solution. DOHA improves $HR$, $HRV$, and $RF$ estimation across unseen-domain protocols and across both end-to-end and hand-crafted rPPG models, validating its plug-and-play nature. The approach offers a practical route to robust remote physiological measurement across diverse conditions and datasets.

Abstract

Remote photoplethysmography (rPPG) technology has become increasingly popular due to its non-invasive monitoring of various physiological indicators, making it widely applicable in multimedia interaction, healthcare, and emotion analysis. Existing rPPG methods utilize multiple datasets for training to enhance the generalizability of models. However, they often overlook the underlying conflict issues across different datasets, such as (1) label conflict resulting from different phase delays between physiological signal labels and face videos at the instance level, and (2) attribute conflict stemming from distribution shifts caused by head movements, illumination changes, skin types, etc. To address this, we introduce the DOmain-HArmonious framework (DOHA). Specifically, we first propose a harmonious phase strategy to eliminate uncertain phase delays and preserve the temporal variation of physiological signals. Next, we design a harmonious hyperplane optimization that reduces irrelevant attribute shifts and encourages the model's optimization towards a global solution that fits more valid scenarios. Our experiments demonstrate that DOHA significantly improves the performance of existing methods under multiple protocols. Our code is available at https://github.com/SWY666/rPPG-DOHA.

Figures (11)

• Figure 1: Contradictory factors exist in rPPG model training. ①: Label conflicts caused by different pulse transit times (red dashed line) and sampling equipment, which result in the chaotic phase delays between desired rPPG signals and "ground truth" signals among different databases. ②: Irrelevant attribute conflicts raised by diverse external noise (e.g., illumination, head motion). Such diversity and the predominance of the intensity over the rPPG signal make the optimization difficult to cover all involved domains.
• Figure 2: An overview of our proposed method. ①exhibits our harmonious phase strategy on ground truth signal and network output, respectively. $\hat{\mathcal{R}}$ and $\mathcal{R}$ present typical network output and label in the form of self-similarity physiological map. ②: a brief demonstration of Harmonious Hyperplane Optimization. Compared with the traditional optimization rule, it eliminates potential invalid instances (instance $X_N$) and then de-conflicts remaining instances on the parameter hyperplane, so that the optimization can cover more attributes.
• Figure 3: A typical self-similarity physiological map, where a higher value (bright) indicates that the temporal messages between two windows are similar, and vice versa. The "signal" can be a ground truth wave (e.g., BVP signal), or the predicted signal of the rPPG model.
• Figure 4: The gradient of baseline CAN has conflicts among both intra- and inter-dataset. For example, the average gradient of VIPL (subset v1-v6) VIPL-HR and PURE PURE prominently present the negative cosine similarity.
• Figure 5: Potential invalid instances (i.e., those with larger gradient norms, area 3) can be disharmonious with others, leading to worse performance of the rPPG model. The involved training instances are collected from task v2 in VIPL VIPL-HR database, the testing sets are PURE PURE and UBFC UBFC.