Biometric Authentication Based on Enhanced Remote Photoplethysmography Signal Morphology

TL;DR

RPPG signal morphology hidden in facial videos can be used for biometric authentication and the experimental results demonstrate that rPPG signal morphology hidden in facial videos can be used for biometric authentication.

Abstract

Remote photoplethysmography (rPPG) is a non-contact method for measuring cardiac signals from facial videos, offering a convenient alternative to contact photoplethysmography (cPPG) obtained from contact sensors. Recent studies have shown that each individual possesses a unique cPPG signal morphology that can be utilized as a biometric identifier, which has inspired us to utilize the morphology of rPPG signals extracted from facial videos for person authentication. Since the facial appearance and rPPG are mixed in the facial videos, we first de-identify facial videos to remove facial appearance while preserving the rPPG information, which protects facial privacy and guarantees that only rPPG is used for authentication. The de-identified videos are fed into an rPPG model to get the rPPG signal morphology for authentication. In the first training stage, unsupervised rPPG training is performed to get coarse rPPG signals. In the second training stage, an rPPG-cPPG hybrid training is performed by incorporating external cPPG datasets to achieve rPPG biometric authentication and enhance rPPG signal morphology. Our approach needs only de-identified facial videos with subject IDs to train rPPG authentication models. The experimental results demonstrate that rPPG signal morphology hidden in facial videos can be used for biometric authentication. The code is available at https://github.com/zhaodongsun/rppg_biometrics.

Figures (7)

• Figure 1: (a) rPPG Authentication System. (b) Our method can improve rPPG morphology information. The fiducial points lovisotto2020seeing like the systolic peaks and diastolic peaks are the main subject-specific biometric characteristics in rPPG signals.
• Figure 2: Face de-identification for rPPG biometrics. The facial appearance is obfuscated while rPPG information is retained.
• Figure 3: The diagram of Contrast-Phys-2D (CP2D) for rPPG unsupervised pre-training based on contrastive learning.
• Figure 4: GT cPPG signal and rPPG signal extracted by CP2D. After the first training stage, the rPPG signal has accurate heartbeats but lacks morphology information.
• Figure 5: rPPG-cPPG hybrid training. The rPPG branch and cPPG branch are trained alternatively to utilize external cPPG signals to enhance the rPPG morphology fully.