LGGNet: Learning from Local-Global-Graph Representations for Brain-Computer Interface

TL;DR

Local-global-graph network (LGGNet) is proposed, a novel neurologically inspired graph neural network (GNN), to learn local-global-graph (LGG) representations of electroencephalography (EEG) for brain–computer interface (BCI) and shows that bringing neuroscience prior knowledge into neural network design yields an improvement of classification performance.

Abstract

Neuropsychological studies suggest that co-operative activities among different brain functional areas drive high-level cognitive processes. To learn the brain activities within and among different functional areas of the brain, we propose LGGNet, a novel neurologically inspired graph neural network, to learn local-global-graph representations of electroencephalography (EEG) for Brain-Computer Interface (BCI). The input layer of LGGNet comprises a series of temporal convolutions with multi-scale 1D convolutional kernels and kernel-level attentive fusion. It captures temporal dynamics of EEG which then serves as input to the proposed local and global graph-filtering layers. Using a defined neurophysiologically meaningful set of local and global graphs, LGGNet models the complex relations within and among functional areas of the brain. Under the robust nested cross-validation settings, the proposed method is evaluated on three publicly available datasets for four types of cognitive classification tasks, namely, the attention, fatigue, emotion, and preference classification tasks. LGGNet is compared with state-of-the-art methods, such as DeepConvNet, EEGNet, R2G-STNN, TSception, RGNN, AMCNN-DGCN, HRNN and GraphNet. The results show that LGGNet outperforms these methods, and the improvements are statistically significant (p<0.05) in most cases. The results show that bringing neuroscience prior knowledge into neural network design yields an improvement of classification performance. The source code can be found at https://github.com/yi-ding-cs/LGG

Figures (7)

• Figure 1: Structure of LGGNet. LGGNet has two main functional blocks: the temporal learning block and the graph learning block. The temporal convolutional layer and the kernel-level attentive fusion layer are shown in the temporal learning block (A). The local and global graph-filtering layers are shown in the graph learning block (B). The temporal convolutional layer aims to learn dynamic temporal representations from EEG directly instead of human extracted features. The kernel-level attentive fusion layer will fuse the information learned by different temporal kernels to increase the learning capacity of LGGNet. The local graph-filtering layer learns the brain activities within each local region. Then the global graph-filtering layer with a trainable adjacency matrix will be applied to learn complex relations among different local regions. Four local graphs are shown in the figure for illustration purposes only, the detailed local-global-graph definitions are provided in 'Defining local-global graphs of EEG' of section III.B. Best viewed in color.
• Figure 2: Three types of local-global-graph definitions. (a) The general local-global-graph definition. This local graph structure is defined according to the 10-20 system. Each local graph reflects the brain activities of a certain brain functional area. (b) The frontal local-global-graph definition. Based on the general local-global graph, the neuroscience evidence of frontal asymmetry patterns in frontal areas is further considered. Six frontal local graphs that are symmetrically located on the left and right frontal areas of the brain are added to learn more discriminative information. (c) The hemisphere local-global-graph definition. The symmetrical local graphs are added for all the functional areas defined in the general local-global graph. The nodes in a local graph are in the same color. The dotted lines are the local graphs. This diagram illustrates the definition for the 62 channel EEG.
• Figure 3: Mean accuracies of LGGNet using different graph structures. The blue bar is the baseline that has no local-global graphs.
• Figure 4: Mean F1 scores of LGGNet using different graph structures. The blue bar is the baseline that has no local-global graphs.
• Figure 5: Mean saliency maps of all subjects for three datasets. These mean saliency maps are for: (a) attention, (b) fatigue, (c) emotion, and (d) preference.