Generalization of Video-Based Heart Rate Estimation Methods To Low Illumination and Elevated Heart Rates
Bhargav Acharya, William Saakyan, Barbara Hammer, Hanna Drimalla
TL;DR
The paper investigates how well remote photoplethysmography (rPPG) methods generalize to low illumination and elevated heart rates by introducing the CHILL dataset and evaluating four classical and four deep learning-based rPPG approaches on CHILL as well as public datasets COHFACE and PURE. It reveals that while DL methods often outperform classical approaches on public data, classical methods can outperform DL on CHILL, and that high heart rates generally reduce accuracy across methods. The study underscores the importance of testing across diverse conditions and highlights that model generalization is strongly influenced by training data composition. The CHILL dataset thus provides a valuable benchmark for evaluating rPPG robustness in realistic, challenging environments, guiding future development and deployment in applications such as telehealth and affective computing.
Abstract
Heart rate is a physiological signal that provides information about an individual's health and affective state. Remote photoplethysmography (rPPG) allows the estimation of this signal from video recordings of a person's face. Classical rPPG methods make use of signal processing techniques, while recent rPPG methods utilize deep learning networks. Methods are typically evaluated on datasets collected in well-lit environments with participants at resting heart rates. However, little investigation has been done on how well these methods adapt to variations in illumination and heart rate. In this work, we systematically evaluate representative state-of-the-art methods for remote heart rate estimation. Specifically, we evaluate four classical methods and four deep learning-based rPPG estimation methods in terms of their generalization ability to changing scenarios, including low lighting conditions and elevated heart rates. For a thorough evaluation of existing approaches, we collected a novel dataset called CHILL, which systematically varies heart rate and lighting conditions. The dataset consists of recordings from 45 participants in four different scenarios. The video data was collected under two different lighting conditions (high and low) and normal and elevated heart rates. In addition, we selected two public datasets to conduct within- and cross-dataset evaluations of the rPPG methods. Our experimental results indicate that classical methods are not significantly impacted by low-light conditions. Meanwhile, some deep learning methods were found to be more robust to changes in lighting conditions but encountered challenges in estimating high heart rates. The cross-dataset evaluation revealed that the selected deep learning methods underperformed when influencing factors such as elevated heart rates and low lighting conditions were not present in the training set.