A Novel Transformer-Based Self-Supervised Learning Method to Enhance Photoplethysmogram Signal Artifact Detection

TL;DR

This work tackles the challenge of detecting motion artifacts in pediatric PPG signals under limited labeled data. It introduces a two-stage framework that pre-trains a Transformer encoder on unlabeled PPG data using self-supervised tasks (Masking, Contrastive learning, and DINO) and then fine-tunes on a small labeled set, aiming to maximize representation quality with few annotations. A key contribution is the Smooth InfoNCE contrastive loss, designed to stabilize training and improve convergence in data-scarce settings. Empirical results show SSL-augmented Transformers substantially outperform fully supervised baselines, with contrastive and DINO SSL approaches delivering the strongest gains, demonstrating practical potential for robust artifact detection in PICU environments with limited annotations. The findings support broader adoption of SSL in clinical signal processing, enabling more reliable downstream decisions when annotated data are scarce.

Abstract

Recent research at CHU Sainte Justine's Pediatric Critical Care Unit (PICU) has revealed that traditional machine learning methods, such as semi-supervised label propagation and K-nearest neighbors, outperform Transformer-based models in artifact detection from PPG signals, mainly when data is limited. This study addresses the underutilization of abundant unlabeled data by employing self-supervised learning (SSL) to extract latent features from these data, followed by fine-tuning on labeled data. Our experiments demonstrate that SSL significantly enhances the Transformer model's ability to learn representations, improving its robustness in artifact classification tasks. Among various SSL techniques, including masking, contrastive learning, and DINO (self-distillation with no labels)-contrastive learning exhibited the most stable and superior performance in small PPG datasets. Further, we delve into optimizing contrastive loss functions, which are crucial for contrastive SSL. Inspired by InfoNCE, we introduce a novel contrastive loss function that facilitates smoother training and better convergence, thereby enhancing performance in artifact classification. In summary, this study establishes the efficacy of SSL in leveraging unlabeled data, particularly in enhancing the capabilities of the Transformer model. This approach holds promise for broader applications in PICU environments, where annotated data is often limited.

Figures (6)

• Figure 1: An end-end process diagram workflow demonstration for the proposed two-stage framework for PPG signal analysis. Step 1 involves pre-training the encoder on unannotated data using different self-supervised learning techniques (masking, contrastive, and DINO) to learn optimal hidden representations. Step 2 fine-tunes the pre-trained encoder with a classifier using annotated data to optimize the classification loss, thereby improving model performance with minimal labeled data.
• Figure 2: Classifier's performance with different learning frameworks including fully-supervised, self-supervised learning with masking, contrastive, and DINO.
• Figure 3: Transformer's performance with different learning frameworks including fully-supervised, self-supervised learning with masking, contrastive, and DINO.
• Figure 4: Contrastive self-supervised learning with different contrastive losses for Transformer.
• Figure 5: Training loss with InfoNCE (left) and SmoothInfoNCE (right) contrastive loss regarding to batches and epochs.