Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Knowledge-guided EEG Representation Learning

Aditya Kommineni, Kleanthis Avramidis, Richard Leahy, Shrikanth Narayanan

TL;DR

The paper addresses the challenge of learning robust EEG representations with limited labels by combining a knowledge-guided objective with a state-space S4 backbone. It introduces a frequency-band power loss grounded in EEG domain knowledge and demonstrates improved downstream performance on motor imagery tasks, along with significant data efficiency gains. The approach achieves competitive accuracy with far fewer parameters than transformer baselines and shows robustness to reduced pre-training and fine-tuning data. This work advances EEG SSL by integrating domain knowledge into self-supervised objectives and points toward broader applicability across biosignals and channel configurations.

Abstract

Self-supervised learning has produced impressive results in multimedia domains of audio, vision and speech. This paradigm is equally, if not more, relevant for the domain of biosignals, owing to the scarcity of labelled data in such scenarios. The ability to leverage large-scale unlabelled data to learn robust representations could help improve the performance of numerous inference tasks on biosignals. Given the inherent domain differences between multimedia modalities and biosignals, the established objectives for self-supervised learning may not translate well to this domain. Hence, there is an unmet need to adapt these methods to biosignal analysis. In this work we propose a self-supervised model for EEG, which provides robust performance and remarkable parameter efficiency by using state space-based deep learning architecture. We also propose a novel knowledge-guided pre-training objective that accounts for the idiosyncrasies of the EEG signal. The results indicate improved embedding representation learning and downstream performance compared to prior works on exemplary tasks. Also, the proposed objective significantly reduces the amount of pre-training data required to obtain performance equivalent to prior works.

Knowledge-guided EEG Representation Learning

TL;DR

The paper addresses the challenge of learning robust EEG representations with limited labels by combining a knowledge-guided objective with a state-space S4 backbone. It introduces a frequency-band power loss grounded in EEG domain knowledge and demonstrates improved downstream performance on motor imagery tasks, along with significant data efficiency gains. The approach achieves competitive accuracy with far fewer parameters than transformer baselines and shows robustness to reduced pre-training and fine-tuning data. This work advances EEG SSL by integrating domain knowledge into self-supervised objectives and points toward broader applicability across biosignals and channel configurations.

Abstract

Self-supervised learning has produced impressive results in multimedia domains of audio, vision and speech. This paradigm is equally, if not more, relevant for the domain of biosignals, owing to the scarcity of labelled data in such scenarios. The ability to leverage large-scale unlabelled data to learn robust representations could help improve the performance of numerous inference tasks on biosignals. Given the inherent domain differences between multimedia modalities and biosignals, the established objectives for self-supervised learning may not translate well to this domain. Hence, there is an unmet need to adapt these methods to biosignal analysis. In this work we propose a self-supervised model for EEG, which provides robust performance and remarkable parameter efficiency by using state space-based deep learning architecture. We also propose a novel knowledge-guided pre-training objective that accounts for the idiosyncrasies of the EEG signal. The results indicate improved embedding representation learning and downstream performance compared to prior works on exemplary tasks. Also, the proposed objective significantly reduces the amount of pre-training data required to obtain performance equivalent to prior works.
Paper Structure (15 sections, 4 equations, 5 figures, 2 tables)

This paper contains 15 sections, 4 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Datasets
  3. Pre-training Dataset
  4. Downstream Datasets
  5. Knowledge-Guided Objectives
  6. Model Architecture
  7. Vanilla S4
  8. Knowledge-guided S4
  9. Experiments
  10. Results
  11. Downstream Performance
  12. Data Efficiency
  13. Pre-training Data
  14. Fine-tuning Data
  15. Conclusion

Figures (5)

  • Figure 1: Montage of 19 EEG channels used for pre-training from TUEG corpus
  • Figure 2: Pre-processing pipeline used for pre-training and fine-tuning datasets
  • Figure 3: Illustration of our knowledge-guided objective which leverages top-down knowledge about signals, enabling learning of robust representations
  • Figure 4: Comparison between model architecture and SSL objectives for proposed pre-trained models. (Left) Vanilla S4 is trained on reconstruction loss. (Right) Knowledge-guided S4 employs an L1 loss between actual and estimated frequency band power in addition to reconstruction loss.
  • Figure 5: Classification performance on BCI IV 2A dataset with decreasing fine-tuning data percentage. Results are illustrated for 100, 50, 30 and 10%.