ARNN: Attentive Recurrent Neural Network for Multi-channel EEG Signals to Identify Epileptic Seizures

TL;DR

An Attention Recurrent Neural Network (ARNN) is proposed that can process a large amount of data efficiently and accurately and inherits the advantages of attention layers and LSTM gate while avoiding their respective drawbacks.

Abstract

Electroencephalography (EEG) is a widely used tool for diagnosing brain disorders due to its high temporal resolution, non-invasive nature, and affordability. Manual analysis of EEG is labor-intensive and requires expertise, making automatic EEG interpretation crucial for reducing workload and accurately assessing seizures. In epilepsy diagnosis, prolonged EEG monitoring generates extensive data, often spanning hours, days, or even weeks. While machine learning techniques for automatic EEG interpretation have advanced significantly in recent decades, there remains a gap in its ability to efficiently analyze large datasets with a balance of accuracy and computational efficiency. To address the challenges mentioned above, an Attention Recurrent Neural Network (ARNN) is proposed that can process a large amount of data efficiently and accurately. This ARNN cell recurrently applies attention layers along a sequence and has linear complexity with the sequence length and leverages parallel computation by processing multi-channel EEG signals rather than single-channel signals. In this architecture, the attention layer is a computational unit that efficiently applies self-attention and cross-attention mechanisms to compute a recurrent function over a wide number of state vectors and input signals. This framework is inspired in part by the attention layer and long short-term memory (LSTM) cells, but it scales this typical cell up by several orders to parallelize for multi-channel EEG signals. It inherits the advantages of attention layers and LSTM gate while avoiding their respective drawbacks. The model's effectiveness is evaluated through extensive experiments with heterogeneous datasets, including the CHB-MIT and UPenn and Mayo's Clinic datasets.

Figures (8)

• Figure 1: Comprehensive illustration of the Attentive recurrent neural network (ARNN) architecture. The architecture begins by segmenting multi-channel input EEG signals into number of local windows, enabling efficient capture of short-term dependencies. Each window undergoes linear projection to generate query (Q), key (K), and value (V) vectors, serving as inputs for the self-attention layer, which captures intra-window dependencies. The input cross-attention mechanism establishes a connection between the current input vectors and recurrent states, facilitating context-aware feature refinement. The state cross-attention layer further maintains interdependencies among state vectors across multiple windows, ensuring temporal continuity. These attention outputs are concatenated and transformed via a linear projection layer to produce a hidden state. This hidden state is processed by the LSTM-style recurrent gate, which dynamically controls the integration of current and past states, enhancing the model's ability to capture both local and global dependencies.
• Figure 2: An illustration of sequence processing by RNN, ViT and ARNN cell. In contrast to sequential processing by RNN and parallel processing by ViT, ARNN cell operates sequentially on multi-dimensional local window of the sequence to produce global dependencies formed by local dependencies.
• Figure 3: Comprehensively description of LSTM-style recurrent gate architecture designed in ARNN cell.
• Figure 4: An illustration depicting the effect of the time step on the (a) training time and (b) accuracy of the ARNN cell at the segment length of 1024 of multi-channel EEG signals. The grey shadow indicates the optimal time step for the cell.
• Figure 5: Testing accuracy of the proposed model against the baseline models across epochs.