Contrastive random lead coding for channel-agnostic self-supervision of biosignals

TL;DR

This work introduces contrastive random lead coding (CRLC), where random subsets of the input channels are used to create positive pairs and compare with using augmentations and neighboring segments in time as positive pairs, which helps generalization across variable channel setups when training the channel-agnostic model.

Abstract

Contrastive learning yields impressive results for self-supervision in computer vision. The approach relies on the creation of positive pairs, something which is often achieved through augmentations. However, for multivariate time series effective augmentations can be difficult to design. Additionally, the number of input channels for biosignal datasets often varies from application to application, limiting the usefulness of large self-supervised models trained with specific channel configurations. Motivated by these challenges, we set out to investigate strategies for creation of positive pairs for channel-agnostic self-supervision of biosignals. We introduce contrastive random lead coding (CRLC), where random subsets of the input channels are used to create positive pairs and compare with using augmentations and neighboring segments in time as positive pairs. We validate our approach by pre-training models on EEG and ECG data, and then fine-tuning them for downstream tasks. CRLC outperforms competing strategies in both scenarios in the channel-agnostic setting. For EEG, the approach additionally outperforms the state-of-the-art reference model. Notably, for EEG tasks CRLC surpasses the current state-of-the-art reference model. While, the state-of-the-art reference model is superior in the ECG task, incorporating CRLC allows us to obtain comparable results. In conclusion, CRLC helps generalization across variable channel setups when training our channel-agnostic model.

Figures (9)

• Figure 1: Caption
• Figure 2: Channel-agnostic setup with the same encoder applied to every channel and a message passing neural network (MPNN) to extract inter-channel interactions. Here, the setup is shown for $C=4$ input channels.
• Figure 3: Generation of positive pairs for contrastive self-supervised learning
• Figure 4: Example of simulated data designed to fit the contrastive random lead coding setting, but not the contrastive segment coding setting.
• Figure 5: Example of simulated data designed to fit the contrastive segment coding setting, but not the contrastive random lead coding setting.