RAICL: Retrieval-Augmented In-Context Learning for Vision-Language-Model Based EEG Seizure Detection

TL;DR

This work addresses the data scarcity and generalization challenges in EEG seizure detection by reframing EEG decoding as a multimodal problem that leverages vision-language models. It introduces RAICL, which retrieves representative resting-state anchors and similar task exemplars from auxiliary subjects and uses chain-of-thought prompts to inject domain expertise, all without retraining the VLMs. Across CHSZ and NICU seizure datasets, the approach with off-the-shelf VLMs achieves competitive or superior performance compared to traditional signal-based methods and image-based baselines, with notable gains from principled exemplar retrieval and CoT prompting. The findings suggest a practical, deployable direction for physiological signal analysis that bridges vision, language, and neural activity, highlighting the primacy of the visual encoder and the potential for extending to multi-class tasks.

Abstract

Electroencephalogram (EEG) decoding is a critical component of medical diagnostics, rehabilitation engineering, and brain-computer interfaces. However, contemporary decoding methodologies remain heavily dependent on task-specific datasets to train specialized neural network architectures. Consequently, limited data availability impedes the development of generalizable large brain decoding models. In this work, we propose a paradigm shift from conventional signal-based decoding by leveraging large-scale vision-language models (VLMs) to analyze EEG waveform plots. By converting multivariate EEG signals into stacked waveform images and integrating neuroscience domain expertise into textual prompts, we demonstrate that foundational VLMs can effectively differentiate between different patterns in the human brain. To address the inherent non-stationarity of EEG signals, we introduce a Retrieval-Augmented In-Context Learning (RAICL) approach, which dynamically selects the most representative and relevant few-shot examples to condition the autoregressive outputs of the VLM. Experiments on EEG-based seizure detection indicate that state-of-the-art VLMs under RAICL achieved better or comparable performance with traditional time series based approaches. These findings suggest a new direction in physiological signal processing that effectively bridges the modalities of vision, language, and neural activities. Furthermore, the utilization of off-the-shelf VLMs, without the need for retraining or downstream architecture construction, offers a readily deployable solution for clinical applications.

Figures (6)

• Figure 1: Three types of EEG-to-image conversion methods. (a) time-frequency spectrogram, using a seizure EEG trial); (b) topographical map, using a P300 EEG trial; and (c) waveform plot, using a seizure EEG trial.
• Figure 2: Overview of the VLM for EEG waveform decoding with RAICL pipeline. The multi-variate EEG time-series is first rendered into a stacked waveform image, and then encoded into high-dimensional embeddings via an image encoder. Certain VLM architectures utilize a projector to map these embeddings into the language model's token space. Native multimodal models treat visual and textual tokens identically within a unified embedding space. For the textual prompt, domain expertise is injected as textual tokens via a CoT framework, as indicated in green. Few-shot examples facilitate ICL in the form of visual tokens, as indicated in red. RAICL strategies introduce metrics of representativeness and similarity in example selection. Finally, the visual and textual token embeddings are concatenated to form a multimodal prompt, querying the language model for autoregressive reasoning and inference.
• Figure 3: Ablation studies of prompt design on two seizure EEG datasets.
• Figure 4: Patient-wise performance in EEG-based seizure detection using Gemini-3-Flash. Ablation studies are conducted on selection strategies of RAICL for the two-shot examples. Each dot denotes the performance for a patient. The red dashed line indicates chance-level performance, and the gray dashed lines in the violin denote quartiles/median. (a) CHSZ dataset; and (b) NICU dataset.
• Figure 5: $t$-SNE visualization of RAICL selection strategies, with both representativeness + similarity, given $M=2$ (two-shot) examples. Embeddings of EEG waveform images are extracted using the visual encoder on the CHSZ dataset. (a) Seizure test query; (b) Non-seizure test query.