DistilCLIP-EEG: Enhancing Epileptic Seizure Detection Through Multi-modal Learning and Knowledge Distillation

TL;DR

DistilCLIP-EEG introduces a multimodal epilepsy detection framework that fuses EEG signals with descriptive text in a shared latent space using a Conformer-based EEG encoder and a BERT-LP text encoder. It leverages learnable prompts and a CLIP-inspired cross-modal objective, augmented by a knowledge distillation pipeline that transfers knowledge from a large teacher to a compact student, achieving substantial parameter and FLOPs reductions while preserving high accuracy. Across four benchmarks, both teacher and student achieve accuracies above 97% and F1 scores above 0.94, with the student model possessing roughly 58% of the teacher’s parameters. The approach delivers robust, efficient seizure detection suitable for resource-constrained settings and offers interpretability through EEG channel activation maps, highlighting potential clinical utility and real-time deployment potential, while acknowledging limitations and avenues for broader validation and modality expansion.

Abstract

Epilepsy is a prevalent neurological disorder marked by sudden, brief episodes of excessive neuronal activity caused by abnormal electrical discharges, which may lead to some mental disorders. Most existing deep learning methods for epilepsy detection rely solely on unimodal EEG signals, neglecting the potential benefits of multimodal information. To address this, we propose a novel multimodal model, DistilCLIP-EEG, based on the CLIP framework, which integrates both EEG signals and text descriptions to capture comprehensive features of epileptic seizures. The model involves an EEG encoder based on the Conformer architecture as a text encoder, the proposed Learnable BERT (BERT-LP) as prompt learning within the encoders. Both operate in a shared latent space for effective cross-modal representation learning. To enhance efficiency and adaptability, we introduce a knowledge distillation method where the trained DistilCLIP-EEG serves as a teacher to guide a more compact student model to reduce training complexity and time. On the TUSZ, AUBMC, and CHB-MIT datasets, both the teacher and student models achieved accuracy rates exceeding 97%. Across all datasets, the F1-scores were consistently above 0.94, demonstrating the robustness and reliability of the proposed framework. Moreover, the student model's parameter count and model size are approximately 58.1% of those of the teacher model, significantly reducing model complexity and storage requirements while maintaining high performance. These results highlight the potential of our proposed model for EEG-based epilepsy detection and establish a solid foundation for deploying lightweight models in resource-constrained settings.

Figures (9)

• Figure 1: The overall architecture of the proposed DistilCLIP-EEG model.
• Figure 2: The network architecture of Conformer processes multi-channel EEG signals as input, where B represents the batch size, C denotes the number of channels, E indicates the number of electrodes, and L stands for the number of signal sampling points.
• Figure 3: (a) Original BERT; (b) The proposed BERT-LP with learnable prompt
• Figure 4: Electrode distribution map of the international 10-20 system for EEG electrode placement.
• Figure 5: Training accuracy and loss curves of the teacher and student models on four EEG datasets.