CuPID: Leveraging Masked Single-Lead ECG Modelling for Enhancing the Representations

TL;DR

CuPID addresses the challenge of learning robust representations from single-lead ECG by mitigating the lack of cross-lead context in masked data modelling. It does so by cueing the decoder with spectrogram-derived context as the Key in a transformer attention mechanism during pre-training, forcing the encoder to produce more informative patch representations that transfer to downstream tasks. Across MIT-AFIB, LT-AF, and Physionet 2017, CuPID outperforms strong single-lead SSL baselines in linear probing and shows competitive gains under fine-tuning, validating the practical value of spectrogram-guided context in ECG representation learning. The findings highlight the potential of context-aware pre-training to enhance remote cardiovascular monitoring while identifying limitations tied to dataset diversity and pre-training data availability.

Abstract

Wearable sensing devices, such as Electrocardiogram (ECG) heart-rate monitors, will play a crucial role in the future of digital health. This continuous monitoring leads to massive unlabeled data, incentivizing the development of unsupervised learning frameworks. While Masked Data Modelling (MDM) techniques have enjoyed wide use, their direct application to single-lead ECG data is suboptimal due to the decoder's difficulty handling irregular heartbeat intervals when no contextual information is provided. In this paper, we present Cueing the Predictor Increments the Detailing (CuPID), a novel MDM method tailored to single-lead ECGs. CuPID enhances existing MDM techniques by cueing spectrogram-derived context to the decoder, thus incentivizing the encoder to produce more detailed representations. This has a significant impact on the encoder's performance across a wide range of different configurations, leading CuPID to outperform state-of-the-art methods in a variety of downstream tasks.

Figures (5)

• Figure 1: Example of the commonly used masking strategy proposed in stmem for 12-Lead ECG processing. Unmasked portions are displayed in blue. Most of the ECG waves remain unmasked in at least one of the leads.
• Figure 2: Reconstruction comparison between CuPID and MTAE. Figures \ref{['fig:baseline_rec']} and \ref{['fig:cupid_rec']} display a reconstruction from MTAE and CuPID, where the unmasked part, the ground truth, and the inference computed by both methods are represented in gray, blue, and orange, respectively. Figure \ref{['fig:loss']} illustrates loss function evolution across pre-training. Figure \ref{['fig:hrv']} compares the performance of CuPID and MTAE, across different heartbeat irregularity levels, measured in SDNN (Standard Deviation of Normal Intervals).
• Figure 3: CuPID architecture. The left side of the Figure shows how the spectrogram is incorporated into the decoder's attention mechanism. This incorporation sets CuPID apart from the standard framework for MDM. The encoder is the model used to address the downstream tasks, while the decoder is discarded after the pre-training. Therefore, this spectrogram is not provided during the evaluation. The right side of the diagram provides a closer look at CuPID's decoder. Due to the challenges of using the spectrogram as a Key, the spectrogram is incorporated from the second block of the decoder. Its first block mirrors the standard decoder block for MDM framework.
• Figure 4: Evolution of the performance over training procedure
• Figure 5: Impact of the model size on the model.