Brant-X: A Unified Physiological Signal Alignment Framework

TL;DR

Brant-X tackles the challenge of modeling correlations between EEG and other physiological signals (EXG) under data scarcity and signal heterogeneity. It leverages the Brant-2 EEG foundation model and introduces a two-level (patch- and sequence-level) contrastive alignment to transfer EEG representations to EXG, aided by sampling augmentation for rate-robustness. The approach achieves state-of-the-art results across sleep stage classification, emotion recognition, FoG detection, and eye movement communication, and is complemented by ablation and case-study analyses that confirm the effectiveness of each component. This work demonstrates a scalable pathway for extending EEG-derived knowledge to other physiological signals, enabling improved multimodal health monitoring and assistive technologies.

Abstract

Physiological signals serve as indispensable clues for understanding various physiological states of human bodies. Most existing works have focused on a single type of physiological signals for a range of application scenarios. However, as the body is a holistic biological system, the inherent interconnection among various physiological data should not be neglected. In particular, given the brain's role as the control center for vital activities, electroencephalogram (EEG) exhibits significant correlations with other physiological signals. Therefore, the correlation between EEG and other physiological signals holds potential to improve performance in various scenarios. Nevertheless, achieving this goal is still constrained by several challenges: the scarcity of simultaneously collected physiological data, the differences in correlations between various signals, and the correlation differences between various tasks. To address these issues, we propose a unified physiological signal alignment framework, Brant-X, to model the correlation between EEG and other signals. Our approach (1) employs the EEG foundation model to data-efficiently transfer the rich knowledge in EEG to other physiological signals, and (2) introduces the two-level alignment to fully align the semantics of EEG and other signals from different semantic scales. In the experiments, Brant-X achieves state-of-the-art performance compared with task-agnostic and task-specific baselines on various downstream tasks in diverse scenarios, including sleep stage classification, emotion recognition, freezing of gaits detection, and eye movement communication. Moreover, the analysis on the arrhythmia detection task and the visualization in case study further illustrate the effectiveness of Brant-X in the knowledge transfer from EEG to other physiological signals. The model's homepage is at https://github.com/zjunet/Brant-X/.

Figures (6)

• Figure 1: Illustration of inherent correlations between EEG and other physiological signals. (a) The waveform patterns in EEG and EOG vary with different sleep stages, especially with the REM stage, which is marked by rapid oscillations in EOG. (b) Excitement boosts heart beats in ECG with enhanced $\beta$ waves evident in EEG. Sadness slows heart rate and increases the brain activity in low-frequency $\alpha$ band. During relaxation, ECG presents a stable heart rate with heightened high-frequency EEG $\theta$ waves.
• Figure 2: Overview of the physiological signal alignment framework Brant-X. Firstly, based on the EEG foundation model, the EXG encoder is trained by the alignment between simultaneously collected EEG and EXG data. Then, the EEG and EXG encoders, capable of learning strong representations from EEG and EXG signals, are applied to various downstream tasks in diverse scenarios.
• Figure 3: Architecture of Brant-X. In the data preparation stage, EXG data are upsampled and downsampled for data augmentation. Then, EEG and EXG data are fed into the EEG encoder and EXG encoder respectively to obtain the representations of data patches. Finally, we align the simultaneously collected EEG and EXG patches and the corresponding EEG and EXG sequences by two-level alignment.
• Figure 4: Overall performance comparison on various tasks.
• Figure 5: Results of the ablation study on all the downstream tasks.

Theorems & Definitions (1)

• definition 1