BTS-rPPG: Orthogonal Butterfly Temporal Shifting for Remote Photoplethysmography

Abstract

Remote photoplethysmography (rPPG) enables contactless physiological sensing from facial videos by analyzing subtle appearance variations induced by blood circulation. However, modeling the temporal dynamics of these signals remains challenging, as many deep learning methods rely on temporal shifting or convolutional operators that aggregate information primarily from neighboring frames, resulting in predominantly local temporal modeling and limited temporal receptive fields. To address this limitation, we propose BTS-rPPG, a temporal modeling framework based on Orthogonal Butterfly Temporal Shifting (BTS). Inspired by the butterfly communication pattern in the Fast Fourier Transform (FFT), BTS establishes structured frame interactions via an XOR-based butterfly pairing schedule, progressively expanding the temporal receptive field and enabling efficient propagation of information across distant frames. Furthermore, we introduce an orthogonal feature transfer mechanism (OFT) that filters the source feature with respect to the target context before temporal shifting, retaining only the orthogonal component for cross-frame transmission. This reduces redundant feature propagation and encourages complementary temporal interaction. Extensive experiments on multiple benchmark datasets demonstrate that BTS-rPPG improves long-range temporal modeling of physiological dynamics and consistently outperforms existing temporal modeling strategies for rPPG estimation.

Figures (4)

• Figure 1: Comparison of temporal communication patterns between (a) existing methods and (b) the proposed BTS-rPPG. Existing methods mainly exchange features between adjacent frames, limiting long-range temporal interaction. In contrast, BTS-rPPG employs a butterfly-inspired hierarchical communication scheme, analogous to FFT-style sparse multi-stage mixing, to progressively expand temporal interaction from local to global scales. This structure is well suited to rPPG signals, which exhibit both short-term continuity and longer-range periodic dependency.
• Figure 2: Overview of the proposed BTS-rPPG. I) The framework comprises (a) an input representation module, (b) a BTS-enhanced Swin Transformer backbone, and (c) a predictor head. RGB frames and their normalized difference frames are fused and embedded into patch-level tokens, which are then processed by Swin Transformer stages equipped with BTS and finally regressed to the target rPPG waveform. II) OFT suppresses redundant transferred information by preserving only the feature component orthogonal to the target context, thereby promoting complementary temporal exchange. III) BTS establishes a butterfly-inspired hierarchical communication pattern, where temporal interactions are progressively expanded across stages to enable efficient propagation from short-range to long-range dependencies.
• Figure 3: Ablation study on intra-dataset MMPD mmpd. This experiment aims to examine how OFT and BTS each contribute to improving the baseline TCS tsm framework.
• Figure 4: Sensitivity analysis of the OFT fold ratio on (a) PURE pure and (b) UBFC-rPPG ubfc datasets. We report MAE and RMSE, and find that the best performance is achieved at $C_s/d=1/4$, which provides sufficient cross-frame transfer while retaining adequate target-specific representation.