EEGDiR: Electroencephalogram denoising network for temporal information storage and global modeling through Retentive Network

TL;DR

This work tackles EEG denoising by leveraging the Retentive Network (Retnet) architecture, adapted to 1D EEG signals through a novel signal embedding that creates inputs compatible with Retnet's 2D-oriented design. It introduces EEGDiR, a multi-level DiR-based architecture with Multi-Scale Retention to capture long-range temporal dependencies and global structure, aided by patch-based signal embedding. Experiments on EOG, EMG, and semi-simulated SS2016 datasets show EEGDiR achieving the best denoising performance, indicated by the lowest $RRMSE_{t}$ and $RRMSE_{s}$ and highest $CC$ across noise levels, outperforming SCNN, 1D-ResCNN, LSTM, and EEGDnet. The study also provides an open-source, preprocessed dataset to standardize benchmarking, highlighting the potential of Retentive networks for EEG processing and broader temporal-signal denoising tasks.

Abstract

Electroencephalogram (EEG) signals play a pivotal role in clinical medicine, brain research, and neurological disease studies. However, susceptibility to various physiological and environmental artifacts introduces noise in recorded EEG data, impeding accurate analysis of underlying brain activity. Denoising techniques are crucial to mitigate this challenge. Recent advancements in deep learningbased approaches exhibit substantial potential for enhancing the signal-to-noise ratio of EEG data compared to traditional methods. In the realm of large-scale language models (LLMs), the Retentive Network (Retnet) infrastructure, prevalent for some models, demonstrates robust feature extraction and global modeling capabilities. Recognizing the temporal similarities between EEG signals and natural language, we introduce the Retnet from natural language processing to EEG denoising. This integration presents a novel approach to EEG denoising, opening avenues for a profound understanding of brain activities and accurate diagnosis of neurological diseases. Nonetheless, direct application of Retnet to EEG denoising is unfeasible due to the one-dimensional nature of EEG signals, while natural language processing deals with two-dimensional data. To facilitate Retnet application to EEG denoising, we propose the signal embedding method, transforming one-dimensional EEG signals into two dimensions for use as network inputs. Experimental results validate the substantial improvement in denoising effectiveness achieved by the proposed method.

Figures (11)

• Figure 1: The diagram illustrates the training procedure of the deep learning method. It includes the Data Processing stage, where raw EEG data and various artifacts undergo preprocessing to create a suitable dataset for deep learning. The dataset comprises sample pairs, namely Noisy EEG signal and Noise-free EEG signal, representing EEG signals with and without noise, respectively. Throughout the training process, the Noisy EEG signal serves as the network input. The network, in turn, produces the Denoised EEG signal, which is utilized as the input for the subsequent training steps. The Loss Function calculates the disparity between the Denoised EEG signal and the Noise-free EEG signal, facilitating the optimization of the network. The black dashed box delineates the inference phase of the network, omitting the optimization component. This inference stage can be likened to the end-to-end output where the Noisy EEG signal is input into the network, yielding the Denoised EEG signal as the output.
• Figure 2: (a) illustrates the architecture of the EEGDiR network. This network generates hidden dimensions through Signal Embedding and obtains the output via linear projection and transformation following multi-level DiR Block processing. EEGDiR operates as an end-to-end model, taking a noisy signal as input and producing a noise-free signal, denoted as $\hat{x}$. The DiR Block, depicted in (b), comprises Pre-Norm, Multi-scale Retention, and Residual Connection, with Pre-Norm utilizing Layer Normalization. The Signal Embedding structure, outlined in (c), involves segmenting the input sequence into new sequences based on the patch size. The hidden dimension after reshaping aligns with the patch size, and after linear projection, it matches the final hidden dimension.
• Figure 3: The recorded signals from the first electrode of Participant 1 in SS2016 are displayed in sequence as segments 1 through 4, denoted as (a), (b), (c), and (d) respectively. Each segment comprises 512 samples, with a sampling rate of 200 SPS.
• Figure 4: The SNR distribution of the SS2016 dataset after signal segmentation is depicted in the figure. It's important to highlight that the SNR values are rounded up for statistical simplicity. The horizontal axis represents various SNR levels ranging from -10 to 20, while the vertical axis indicates the number of segments corresponding to each SNR level.
• Figure 5: Performance of four deep-learning networks at different SNR levels with EOG dataset artifact removal. The smaller $RRMSE_{t}$ and $RRMSE_{s}$ , and the larger Correlation Coefficient($CC$), the better denoising effect. The denoising performance increases as the SNR increases.