Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MHANet: Multi-scale Hybrid Attention Network for Auditory Attention Detection

Lu Li, Cunhang Fan, Hongyu Zhang, Jingjing Zhang, Xiaoke Yang, Jian Zhou, Zhao Lv

TL;DR

MHANet tackles auditory attention decoding from EEG in noisy multi-speaker environments by introducing a two-block architecture: a Multi-scale Hybrid Attention (MHA) module and a Spatiotemporal Convolution (STC) module. The MHA combines channel attention with a Multi-scale Temporal Attention (MTA) and a Multi-scale Global Attention (MGA) to capture long-short range spatiotemporal dependencies, while STC aggregates these features through temporal and spatial convolutions with pooling. This approach achieves state-of-the-art decoding accuracy on KUL, DTU, and AVED datasets, including a strong 0.1-second decision window performance, with only 0.02M trainable parameters, highlighting excellent efficiency. The work demonstrates the importance of multi-scale, globally aware attention for EEG-based AAD and points to potential future gains from integrating time-frequency analyses for further improvements in real-time hearing aid applications.

Abstract

Auditory attention detection (AAD) aims to detect the target speaker in a multi-talker environment from brain signals, such as electroencephalography (EEG), which has made great progress. However, most AAD methods solely utilize attention mechanisms sequentially and overlook valuable multi-scale contextual information within EEG signals, limiting their ability to capture long-short range spatiotemporal dependencies simultaneously. To address these issues, this paper proposes a multi-scale hybrid attention network (MHANet) for AAD, which consists of the multi-scale hybrid attention (MHA) module and the spatiotemporal convolution (STC) module. Specifically, MHA combines channel attention and multi-scale temporal and global attention mechanisms. This effectively extracts multi-scale temporal patterns within EEG signals and captures long-short range spatiotemporal dependencies simultaneously. To further improve the performance of AAD, STC utilizes temporal and spatial convolutions to aggregate expressive spatiotemporal representations. Experimental results show that the proposed MHANet achieves state-of-the-art performance with fewer trainable parameters across three datasets, 3 times lower than that of the most advanced model. Code is available at: https://github.com/fchest/MHANet.

MHANet: Multi-scale Hybrid Attention Network for Auditory Attention Detection

TL;DR

MHANet tackles auditory attention decoding from EEG in noisy multi-speaker environments by introducing a two-block architecture: a Multi-scale Hybrid Attention (MHA) module and a Spatiotemporal Convolution (STC) module. The MHA combines channel attention with a Multi-scale Temporal Attention (MTA) and a Multi-scale Global Attention (MGA) to capture long-short range spatiotemporal dependencies, while STC aggregates these features through temporal and spatial convolutions with pooling. This approach achieves state-of-the-art decoding accuracy on KUL, DTU, and AVED datasets, including a strong 0.1-second decision window performance, with only 0.02M trainable parameters, highlighting excellent efficiency. The work demonstrates the importance of multi-scale, globally aware attention for EEG-based AAD and points to potential future gains from integrating time-frequency analyses for further improvements in real-time hearing aid applications.

Abstract

Auditory attention detection (AAD) aims to detect the target speaker in a multi-talker environment from brain signals, such as electroencephalography (EEG), which has made great progress. However, most AAD methods solely utilize attention mechanisms sequentially and overlook valuable multi-scale contextual information within EEG signals, limiting their ability to capture long-short range spatiotemporal dependencies simultaneously. To address these issues, this paper proposes a multi-scale hybrid attention network (MHANet) for AAD, which consists of the multi-scale hybrid attention (MHA) module and the spatiotemporal convolution (STC) module. Specifically, MHA combines channel attention and multi-scale temporal and global attention mechanisms. This effectively extracts multi-scale temporal patterns within EEG signals and captures long-short range spatiotemporal dependencies simultaneously. To further improve the performance of AAD, STC utilizes temporal and spatial convolutions to aggregate expressive spatiotemporal representations. Experimental results show that the proposed MHANet achieves state-of-the-art performance with fewer trainable parameters across three datasets, 3 times lower than that of the most advanced model. Code is available at: https://github.com/fchest/MHANet.

Paper Structure

This paper contains 23 sections, 17 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. The proposed MHANet
  3. Overall Architecture
  4. Multi-scale Hybrid Attention Module
  5. Multi-scale Temporal Attention Block
  6. Multi-scale Global Attention Block
  7. Spatiotemporal Convolution Module
  8. Experiments
  9. Datasets
  10. Data Processing
  11. Implementation Details
  12. Results on AAD
  13. Performance of MHANet
  14. Ablation Experiment
  15. Analysis and Discussion
  16. ...and 8 more sections

Figures (1)

  • Figure 1: The overview architecture of our MHANet model for AAD, which mainly consists of two modules: (a) multi-scale hybrid attention (MHA) module and (b) spatiotemporal convolution (STC) module.