RhythmMamba: Fast, Lightweight, and Accurate Remote Physiological Measurement

TL;DR

RhythmMamba introduces a fast, lightweight, and accurate rPPG method by reframing the task as time-series modeling with a state-space approach. A frame stem embeds spatial information into channels, while a multi-temporal constraint Mamba captures long-range periodic patterns and a frequency-domain FFN enables inter-channel spectral interactions, achieving state-of-the-art results with high throughput and low memory footprint. The approach demonstrates strong intra- and cross-dataset performance and supports arbitrary-length inputs, offering practical benefits for mobile and real-world deployment. Together, these components address the core challenges of long-range dependency and spatiotemporal redundancy in rPPG with linear complexity and robust generalization.

Abstract

Remote photoplethysmography (rPPG) is a method for non-contact measurement of physiological signals from facial videos, holding great potential in various applications such as healthcare, affective computing, and anti-spoofing. Existing deep learning methods struggle to address two core issues of rPPG simultaneously: understanding the periodic pattern of rPPG among long contexts and addressing large spatiotemporal redundancy in video segments. These represent a trade-off between computational complexity and the ability to capture long-range dependencies. In this paper, we introduce RhythmMamba, a state space model-based method that captures long-range dependencies while maintaining linear complexity. By viewing rPPG as a time series task through the proposed frame stem, the periodic variations in pulse waves are modeled as state transitions. Additionally, we design multi-temporal constraint and frequency domain feed-forward, both aligned with the characteristics of rPPG time series, to improve the learning capacity of Mamba for rPPG signals. Extensive experiments show that RhythmMamba achieves state-of-the-art performance with 319% throughput and 23% peak GPU memory. The codes are available at https://github.com/zizheng-guo/RhythmMamba.

Figures (6)

• Figure 1: A schematic diagram of state transitions. Considering the periodic nature of rPPG, the rPPG signal can be represented using a finite number of states. Where $h(t)$ represents the state vector, $x(t)$ represents the input vector, and $y(t)$ represents the output vector.
• Figure 2: Performance and efficiency evaluation for intra-dataset testing on MMPD. The diameter of the circle indicates the peak GPU memory. The proposed RhythmMamba is faster, lighter, and achieves comparable accuracy, with these advantages becoming more pronounced as the scale increases due to its linear complexity.
• Figure 3: The framework of RhythmMamba. It consists of frame stem, multi-temporal constraint Mamba, frequency domain feed-forward, and rPPG predictor head. Where "$+$" represents addition, "$\times$" represents multiplication, "$\sigma$" represents the activation layer, and trapezoid represents the linear layer.
• Figure a: An example of results on MMPD.
• Figure b: Bland-Altman plots of results.