Continual Learning for Remote Physiological Measurement: Minimize Forgetting and Simplify Inference

TL;DR

This work addresses the problem of domain incremental learning for remote physiological measurement (rPPG) by introducing ADDP, a rehearsal-free framework that uses adapter-based finetuning, domain prototypes, and prototype-based augmentation to minimize forgetting across sequential domain shifts. The method also includes a novel prototype-based inference simplification that transfers test samples to the model's most familiar style, improving robustness. The authors establish the first rPPG continual-learning protocol and demonstrate that ADDP outperforms several baselines, achieving competitive final incremental performance while preserving past knowledge with a compact, privacy-friendly setup. The approach offers practical benefits for real-world, privacy-conscious deployment of remote physiological sensing with efficient adaptation to new environments and subjects.

Abstract

Remote photoplethysmography (rPPG) has gained significant attention in recent years for its ability to extract physiological signals from facial videos. While existing rPPG measurement methods have shown satisfactory performance in intra-dataset and cross-dataset scenarios, they often overlook the incremental learning scenario, where training data is presented sequentially, resulting in the issue of catastrophic forgetting. Meanwhile, most existing class incremental learning approaches are unsuitable for rPPG measurement. In this paper, we present a novel method named ADDP to tackle continual learning for rPPG measurement. We first employ adapter to efficiently finetune the model on new tasks. Then we design domain prototypes that are more applicable to rPPG signal regression than commonly used class prototypes. Based on these prototypes, we propose a feature augmentation strategy to consolidate the past knowledge and an inference simplification strategy to convert potentially forgotten tasks into familiar ones for the model. To evaluate ADDP and enable fair comparisons, we create the first continual learning protocol for rPPG measurement. Comprehensive experiments demonstrate the effectiveness of our method for rPPG continual learning. Source code is available at \url{https://github.com/MayYoY/rPPGDIL}

Figures (3)

• Figure 1: (a) Example sequences from popular benchmark rPPG datasets. Samples from different datasets have different backgrounds, skin colors, head motions, lighting conditions, etc. (b) The performance of mainstream rPPG methods on the initial task during the incremental learning process. Previous methods exhibit obvious catastrophic forgetting while our method (ADDP) effectively alleviates this phenomenon.
• Figure 2: The architecture of our model. The base model is Uniformer which can extract the crucial local features for rPPG measurement. Our DiffNorm module can effectively fuse the appearance and dynamic features. Only the adapter and regression modules (the orange blocks) are learnable after the initial task.
• Figure 3: Overview of our domain prototype-based strategies. (a) During training stage, two types of domain prototypes, i.e. style prototypes and noise prototypes, are extracted for each task. Meanwhile, we randomly select domain prototypes of previous tasks to augment the training samples and record the training MAE of the selected style prototypes. (b) In the inference stage, the style prototype with lowest MAE is selected to transfer test samples into a style that can be easily processed.