A Plug-and-Play Temporal Normalization Module for Robust Remote Photoplethysmography

TL;DR

Temporal Normalization is introduced, a flexible plug-and-play module compatible with any end-to-end rPPG network architecture that effectively mitigates motion and lighting artifacts, significantly boosting the rPPG prediction performance.

Abstract

Remote photoplethysmography (rPPG) extracts PPG signals from subtle color changes in facial videos, showing strong potential for health applications. However, most rPPG methods rely on intensity differences between consecutive frames, missing long-term signal variations affected by motion or lighting artifacts, which reduces accuracy. This paper introduces Temporal Normalization (TN), a flexible plug-and-play module compatible with any end-to-end rPPG network architecture. By capturing long-term temporally normalized features following detrending, TN effectively mitigates motion and lighting artifacts, significantly boosting the rPPG prediction performance. When integrated into four state-of-the-art rPPG methods, TN delivered performance improvements ranging from 34.3% to 94.2% in heart rate measurement tasks across four widely-used datasets. Notably, TN showed even greater performance gains in smaller models. We further discuss and provide insights into the mechanisms behind TN's effectiveness.

Figures (6)

• Figure 1: TN module improves rPPG performance. The Plug-and-Play TN module detrend pixels along the temporal dimension and divide by the Standard Deviation (S.D.), demonstrates a strong capacity for noise suppression, leading to an average error reduction of 75% for four baselines (TS-CANmttscan, PhysNetphysnet, EffPhys-Cefficientphys, and PhysFormerphysformer), trained on the MMPDtang2023mmpd dataset.
• Figure 2: Visualization of temporal normalization (TN) module. (a)Without TN, the difference between facial frames and the average RGB values is slight, resulting in noisy prediction. (b)After adding the TN module, the temporal features and the difference of averages are amplified, leading to strong periodic patterns corresponding to the peaks of the BVP waveform.
• Figure 3: TN enhances the pulse signal component and reduces optical noise. According to a real-world optical model, the camera primarily captures stable light along with periodic noise and pulse. Traditional difference methods eliminate the light component but also amplify noise and the pulse. Our TN module effectively weakens the light and noise while enhancing the desired pulse signal.
• Figure 4: TN establishes the global correlation across long frames. The temporal normalization features are integrated through two branches: the local frame features and the global standard deviation features. These jointly compute the normalized features, enhancing the global correlation of the features.
• Figure 5: The influence of model parameters on the performance of the TN module. Due to the effective global correlation of TN module, PhysNet can achieve high performance (average MAE 3.59) with only 10K parameters.