Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BiMoE: Brain-Inspired Experts for EEG-Dominant Affective State Recognition

Hongyu Zhu, Lin Chen, Mingsheng Shang

Abstract

Multimodal Sentiment Analysis (MSA) that integrates Electroencephalogram (EEG) with peripheral physiological signals (PPS) is crucial for the development of brain-computer interface (BCI) systems. However, existing methods encounter three major challenges: (1) overlooking the region-specific characteristics of affective processing by treating EEG signals as homogeneous; (2) treating EEG as a black-box input, which lacks interpretability into neural representations;(3) ineffective fusion of EEG features with complementary PPS features. To overcome these issues, we propose BiMoE, a novel brain-inspired mixture of experts framework. BiMoE partitions EEG signals in a brain-topology-aware manner, with each expert utilizing a dual-stream encoder to extract local and global spatiotemporal features. A dedicated expert handles PPS using multi-scale large-kernel convolutions. All experts are dynamically fused through adaptive routing and a joint loss function. Evaluated under strict subject-independent settings, BiMoE consistently surpasses state-of-the-art baselines across various affective dimensions. On the DEAP and DREAMER datasets, it yields average accuracy improvements of 0.87% to 5.19% in multimodal sentiment classification. The code is available at: https://github.com/HongyuZhu-s/BiMo.

BiMoE: Brain-Inspired Experts for EEG-Dominant Affective State Recognition

Abstract

Multimodal Sentiment Analysis (MSA) that integrates Electroencephalogram (EEG) with peripheral physiological signals (PPS) is crucial for the development of brain-computer interface (BCI) systems. However, existing methods encounter three major challenges: (1) overlooking the region-specific characteristics of affective processing by treating EEG signals as homogeneous; (2) treating EEG as a black-box input, which lacks interpretability into neural representations;(3) ineffective fusion of EEG features with complementary PPS features. To overcome these issues, we propose BiMoE, a novel brain-inspired mixture of experts framework. BiMoE partitions EEG signals in a brain-topology-aware manner, with each expert utilizing a dual-stream encoder to extract local and global spatiotemporal features. A dedicated expert handles PPS using multi-scale large-kernel convolutions. All experts are dynamically fused through adaptive routing and a joint loss function. Evaluated under strict subject-independent settings, BiMoE consistently surpasses state-of-the-art baselines across various affective dimensions. On the DEAP and DREAMER datasets, it yields average accuracy improvements of 0.87% to 5.19% in multimodal sentiment classification. The code is available at: https://github.com/HongyuZhu-s/BiMo.

Paper Structure

This paper contains 15 sections, 18 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Task Definition
  4. Expert Structure
  5. Global–Local Dual-Stream EEG Feature Extraction
  6. PPS Representation and Expert Coordination
  7. Joint Optimization
  8. Experiments
  9. Data Preprocessing
  10. DEAP
  11. DREAMER
  12. Experiment Settings
  13. Verification Performance of BiMoE
  14. Ablation Experiment
  15. Conclusion

Figures (3)

  • Figure 1: Expert structure diagram. EEG signals from different regions of the brain and the PPS are assigned to different expert networks for encoding.
  • Figure 2: (a) Overview of the BiMoE framework. BiMoE primarily comprises: (b) the Global-Local Dual-stream network for EEG processing, (c) the Multi-Scale Large-Kernel Convolutional module for PPS, (d) a Router Network for dynamic feature integration, and a specific joint loss function for optimization.
  • Figure 3: SHAP summary plots for the DEAP (left) and DREAMER (right) datasets.