Refining Remote Photoplethysmography Architectures using CKA and Empirical Methods

TL;DR

This paper addresses how to determine optimal depth for deep rPPG architectures by applying Centered Kernel Alignment (CKA) to two representative models, PhysNet-3DCNN and TS-CAN. By varying depth and analyzing activations with CKA, the authors identify representational redundancies in deeper networks and underfitting in shallower ones, validating these findings with cross-dataset MAE analyses. The results suggest that a mid-range depth (approximately 5 layers for both architectures) captures the essential representations found in deeper models while avoiding unnecessary computation, and that CKA can guide principled architectural refinement beyond brute-force searches. The work demonstrates that CKA-based diagnostics align with empirical performance and offers a roadmap for more efficient, interpretable rPPG model design, with potential applicability to other video-based physiological sensing tasks.

Abstract

Model architecture refinement is a challenging task in deep learning research fields such as remote photoplethysmography (rPPG). One architectural consideration, the depth of the model, can have significant consequences on the resulting performance. In rPPG models that are overprovisioned with more layers than necessary, redundancies exist, the removal of which can result in faster training and reduced computational load at inference time. With too few layers the models may exhibit sub-optimal error rates. We apply Centered Kernel Alignment (CKA) to an array of rPPG architectures of differing depths, demonstrating that shallower models do not learn the same representations as deeper models, and that after a certain depth, redundant layers are added without significantly increased functionality. An empirical study confirms how the architectural deficiencies discovered using CKA impact performance, and we show how CKA as a diagnostic can be used to refine rPPG architectures.

Figures (6)

• Figure 1: Overview of using CKA to inform architecture refinement by revealing similar and dissimilar layers between architectures.
• Figure 2: CKA comparison across augmentations, datasets, and architectures, demonstrating that blocks of layers exhibit similar behavior.
• Figure 3: CKA self-similarity comparison for 3DCNN (\ref{['fig:cka-self:3DCNN']}) and TS-CAN (\ref{['fig:cka-self:TSCAN']}) based architectures on the PURE dataset.
• Figure 4: CKA 10-to-all (\ref{['fig:cka-10cross']}) and 5-to-all (\ref{['fig:cka-5cross']}) cross-similarity comparison for 3DCNN-based architectures on the PURE dataset.
• Figure 5: CKA 2-to-all cross-similarity comparison for TS-CAN based architectures on the PURE dataset.