Learning Domain- and Class-Disentangled Prototypes for Domain-Generalized EEG Emotion Recognition

Abstract

Electroencephalography (EEG)-based emotion recognition plays a critical role in affective Brain-Computer Interfaces (aBCIs), yet its practical deployment remains limited by inter-subject variability, reliance on target-domain data, and unavoidable label noise. To address these challenges, we propose a Multi-domain Aggregation Transfer learning framework with domain-class prototypes (MAT) for emotion recognition under completely unseen target domains. MAT introduces a feature decoupling module to disentangle class-invariant domain features from domain-invariant class features, enabling more robust and interpretable EEG representations. A Hierarchical-Domain Aggregation (HDA) mechanism based on Maximum Mean Discrepancy (MMD) constructs superdomains to model shared distributional structures across subjects, while adaptive prototype updating refines domain and class prototypes to capture stable intrinsic representations. Moreover, a pairwise learning strategy reformulates classification as similarity estimation between sample pairs, effectively mitigating the effect of label noise. Extensive experiments on three public EEG emotion datasets (SEED, SEED-IV, and SEED-V) show that the accuracy of MAT is improved by 2.87%, 3.84%, and 2.05% compared with the state-of-the-art (SOTA) model for unseen target domains. Our results provide a promising direction for emotion recognition under real-world unseen-subject scenarios.The source code is available at https://github.com/WuCB-BCI/MAT.

Figures (10)

• Figure 1: Illustration of the transfer learning paradigm. Knowledge extracted from one or multiple source domains is transferred to a target domain to enhance feature alignment and model generalization across subjects.
• Figure 2: The training phase of the MAT framework. Here, EEG features are first decoupled into domain- and class-specific components through discriminators $D_d$ and $D_c$ with a Gradient Reversal Layer (GRL) for adversarial alignment. Second, domains $S_1\sim S_n$ are clustered into superdomains $S_1\sim S_K$ by HDA mechanism to capture shared yet distinct subject representations. Within each superdomain, domain prototypes $\mu_d$ and class prototypes $\mu_c$ are adaptively updated to ensure stable and discriminative feature learning. Finally, the traditional classification problem was transformed into the problem of similarity between samples, improving robustness against label noise and unseen target domains.
• Figure 3: The inferencing phase of the MAT framework. The optimal domain prototype $\mu_d$ and class prototype $\mu_c$ obtained during the model training phase are transferred to the unseen target domain. Specifically, we first decouple the sample features, and then the model determines the superdomain space $\mu_d$ by Domain Prototype Inference, and perform Class Prototype Inference within this superdomain space to determine the emotion category $\mu_c$. Here, $h(\cdot)$ represent the bilinear transformation to capture the most relevant domain space (Eq.\ref{['Eq:Domain Inference']}). $d_{cos}(\cdot)$ represents the similarity evaluation (Eq.\ref{['Eq:Class Inference']}).
• Figure 4: Confusion matrices of different baseline model under cross-subject single-session LOSO cross-validation. The SEED database (a)$\sim$(d) contains three emotion categories: negative, neutral and positive. The Seed-IV database (e)$\sim$(h) contains four emotion categories: neutral, sad, fear and happy. The horizontal axis represents the predicted labels, while the vertical axis represents the true labels.
• Figure 5: Confusion matrices of different baseline model under cross-subject single-session LOSO cross-validation. The Seed-V database contains five emotion categories: happy, fear, neutral, sad, and disgust. The horizontal axis represents the predicted labels, while the vertical axis represents the true labels.