M3PD Dataset: Dual-view Photoplethysmography (PPG) Using Front-and-rear Cameras of Smartphones in Lab and Clinical Settings

TL;DR

The paper addresses the reliability gap of smartphone-based video photoplethysmography (rPPG) for cardiovascular monitoring by introducing M3PD, the first dual-view smartphone dataset capturing synchronized front-face and rear-fingertip videos in lab and clinical settings from 60 participants (including 47 cardiovascular patients). It proposes F3Mamba, a TD-Mamba/CSSM-based fusion framework that jointly models and fuses facial and fingertip signals to yield robust heart-rate estimates, outperforming single-view baselines by $21.9$–$30.2\%$ in MAE. Across intra- and cross-dataset evaluations, F3Mamba demonstrates strong accuracy and generalization, with Lab→Clinic MAE of $8.204$ BPM (ρ=$0.644$) and Clinic→Lab MAE of $9.360$ BPM (ρ=$0.546$), highlighting practical clinical relevance. The dataset and framework enable broader, multi-parameter mobile health sensing and telemedicine applications, while acknowledging limitations such as still limited demographic diversity and lack of continuous ECG ground truth for beat-level analysis.

Abstract

Portable physiological monitoring is essential for early detection and management of cardiovascular disease, but current methods often require specialized equipment that limits accessibility or impose impractical postures that patients cannot maintain. Video-based photoplethysmography on smartphones offers a convenient noninvasive alternative, yet it still faces reliability challenges caused by motion artifacts, lighting variations, and single-view constraints. Few studies have demonstrated reliable application to cardiovascular patients, and no widely used open datasets exist for cross-device accuracy. To address these limitations, we introduce the M3PD dataset, the first publicly available dual-view mobile photoplethysmography dataset, comprising synchronized facial and fingertip videos captured simultaneously via front and rear smartphone cameras from 60 participants (including 47 cardiovascular patients). Building on this dual-view setting, we further propose F3Mamba, which fuses the facial and fingertip views through Mamba-based temporal modeling. The model reduces heart-rate error by 21.9 to 30.2 percent over existing single-view baselines while improving robustness in challenging real-world scenarios. Data and code: https://github.com/Health-HCI-Group/F3Mamba.

Figures (11)

• Figure 1: Fusion of video-based physiological sensing. Video-based physiological sensing faces challenges from motion artifacts, lighting variations, and position instability. Traditional approaches rely on single views (facial or fingertip), limiting robustness. Our dual-position fusion method integrates signals from both front camera (facial) and rear camera (fingertip) videos. The $\mathrm{F}^3\text{Mamba}$ framework leverages this dual-view approach to enhance algorithm robustness and accuracy in heart rate estimation across real-world scenarios.
• Figure 2: Data collection setup and real-world challenges in dual-view mobile rPPG. (a) Synchronized data acquisition system capturing facial and fingertip videos simultaneously via front and rear smartphone cameras, with concurrent physiological measurements including respiratory sensor, blood pressure monitor, and pulse oximeter. (b) Representative recording samples showing facial videos from elderly cardiovascular patients and fingertip videos with characteristic red appearance from rear camera flash. (c) Facial video challenges during handheld recording: motion artifacts from natural head movements and low-angle perspective distortions common in patient self-monitoring. (d) Fingertip video challenges: finger disattachment from camera surface and lateral finger displacement, particularly prevalent among elderly users with limited dexterity.
• Figure 3: Camera color reproduction variability across devices and environments. Comparison of ColorChecker Classic captured by (a) Xiaomi 14 in clinical settings and (b) OPPO A52 in laboratory settings. The distinct color reproductions reflect differences in camera sensors and image signal processors (ISPs) between devices, demonstrating the hardware variability that algorithms must handle for robust cross-device generalization in real-world mobile health monitoring applications.
• Figure 4: Synchronized multi-modal data acquisition system. The system interface displays real-time physiological waveforms including blood volume pulse (BVP, top) and respiratory rate (RR, bottom) signals synchronized with simultaneous dual-view smartphone recording. The right panel shows the mobile application interface capturing both facial (front camera) and fingertip (rear camera) videos with real-time preview and recording controls.
• Figure 5: Experimental protocol for data collection. The protocol consists of five phases designed to simulate real-world cardiovascular monitoring scenarios: baseline resting state (5 min), breath-holding for autonomic response testing (1 min), recovery period (2 min), high leg lifts for exertional heart rate changes (1 min), and final recovery phase (4 min). Blood pressure measurements were taken during the breath-holding phase to capture comprehensive cardiovascular parameters.