SiamQuality: A ConvNet-Based Foundation Model for Imperfect Physiological Signals

TL;DR

This work tackles the challenge of building foundation models from imperfect physiological data by focusing on data quality in PPG signals. It introduces SiamQuality, a CNN-based framework that employs a SimSiam self-supervised objective with signal-quality pairing and curriculum learning to align representations from good and nearby bad-quality signals. Pre-trained on over 36 million PPG pairs and fine-tuned on six downstream tasks, SiamQuality achieves state-of-the-art performance on several heart-monitoring–related metrics and demonstrates robustness to artifacts. The study also provides practical deployment insights and calls for data-quality–aware design in future biosignal foundations with broader applicability beyond PPG.

Abstract

Foundation models, especially those using transformers as backbones, have gained significant popularity, particularly in language and language-vision tasks. However, large foundation models are typically trained on high-quality data, which poses a significant challenge, given the prevalence of poor-quality real-world data. This challenge is more pronounced for developing foundation models for physiological data; such data are often noisy, incomplete, or inconsistent. The present work aims to provide a toolset for developing foundation models on physiological data. We leverage a large dataset of photoplethysmography (PPG) signals from hospitalized intensive care patients. For this data, we propose SimQuality, a novel self-supervised learning task based on convolutional neural networks (CNNs) as the backbone to enforce representations to be similar for good and poor quality signals that are from similar physiological states. We pre-trained the SimQuality on over 36 million 30-second PPG pairs and then fine-tuned and tested on six downstream tasks using external datasets. The results demonstrate the superiority of the proposed approach on all the downstream tasks, which are extremely important for heart monitoring on wearable devices. Our method indicates that CNNs can be an effective backbone for foundation models that are robust to training data quality.

Figures (6)

• Figure 1: The proposed SiamQuality to address the data quality issue in physiological data.
• Figure 2: The mechanism for signal quality pairing
• Figure 3: Architecture for SimSiam with quality pairing augmentation
• Figure 4: AT-curve. The horizontal axis represents the upper limit of the signal quality for each subgroup. The height of each bar denotes the sample size for each respective subgroup. The line plot shows MAE within each subgroup.
• Figure 5: AT-Curve for all downstream tasks