Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Auditory Attention Decoding without Spatial Information: A Diotic EEG Study

Masahiro Yoshino, Haruki Yokota, Junya Hara, Yuichi Tanaka, Hiroshi Higashi

TL;DR

This work tackles auditory attention decoding (AAD) in diotic, spatially non-informative settings by eliminating interaural cues. It introduces a diotic AAD framework that maps EEG via BrainNetwork and speech via wav2vec 2.0 plus a 2-layer CNN into a shared latent space, scoring attentional alignment with cosine similarity. On a diotic EEG dataset, the proposed model achieves 72.70% accuracy on 5 s segments, substantially outperforming a direction-based baseline (DARNet) at 50.12%, suggesting successful decoding from speech content alone. Cognitive-neuroscience analyses using a Match-Mismatch task and SHAP explainability indicate the model operates at late attentional selection and engages frontal control networks, supporting its neural plausibility. The findings have practical implications for next-generation smart hearing aids and objective audiometry in complex real-world listening environments.

Abstract

Auditory attention decoding (AAD) identifies the attended speech stream in multi-speaker environments by decoding brain signals such as electroencephalography (EEG). This technology is essential for realizing smart hearing aids that address the cocktail party problem and for facilitating objective audiometry systems. Existing AAD research mainly utilizes dichotic environments where different speech signals are presented to the left and right ears, enabling models to classify directional attention rather than speech content. However, this spatial reliance limits applicability to real-world scenarios, such as the "cocktail party" situation, where speakers overlap or move dynamically. To address this challenge, we propose an AAD framework for diotic environments where identical speech mixtures are presented to both ears, eliminating spatial cues. Our approach maps EEG and speech signals into a shared latent space using independent encoders. We extract speech features using wav2vec 2.0 and encode them with a 2-layer 1D convolutional neural network (CNN), while employing the BrainNetwork architecture for EEG encoding. The model identifies the attended speech by calculating the cosine similarity between EEG and speech representations. We evaluate our method on a diotic EEG dataset and achieve 72.70% accuracy, which is 22.58% higher than the state-of-the-art direction-based AAD method.

Auditory Attention Decoding without Spatial Information: A Diotic EEG Study

TL;DR

This work tackles auditory attention decoding (AAD) in diotic, spatially non-informative settings by eliminating interaural cues. It introduces a diotic AAD framework that maps EEG via BrainNetwork and speech via wav2vec 2.0 plus a 2-layer CNN into a shared latent space, scoring attentional alignment with cosine similarity. On a diotic EEG dataset, the proposed model achieves 72.70% accuracy on 5 s segments, substantially outperforming a direction-based baseline (DARNet) at 50.12%, suggesting successful decoding from speech content alone. Cognitive-neuroscience analyses using a Match-Mismatch task and SHAP explainability indicate the model operates at late attentional selection and engages frontal control networks, supporting its neural plausibility. The findings have practical implications for next-generation smart hearing aids and objective audiometry in complex real-world listening environments.

Abstract

Auditory attention decoding (AAD) identifies the attended speech stream in multi-speaker environments by decoding brain signals such as electroencephalography (EEG). This technology is essential for realizing smart hearing aids that address the cocktail party problem and for facilitating objective audiometry systems. Existing AAD research mainly utilizes dichotic environments where different speech signals are presented to the left and right ears, enabling models to classify directional attention rather than speech content. However, this spatial reliance limits applicability to real-world scenarios, such as the "cocktail party" situation, where speakers overlap or move dynamically. To address this challenge, we propose an AAD framework for diotic environments where identical speech mixtures are presented to both ears, eliminating spatial cues. Our approach maps EEG and speech signals into a shared latent space using independent encoders. We extract speech features using wav2vec 2.0 and encode them with a 2-layer 1D convolutional neural network (CNN), while employing the BrainNetwork architecture for EEG encoding. The model identifies the attended speech by calculating the cosine similarity between EEG and speech representations. We evaluate our method on a diotic EEG dataset and achieve 72.70% accuracy, which is 22.58% higher than the state-of-the-art direction-based AAD method.
Paper Structure (24 sections, 6 equations, 5 figures, 1 table)

This paper contains 24 sections, 6 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Diotic Auditory Attention Decoding
  3. Task
  4. Framework
  5. Feature Extraction
  6. Similarity Calculation
  7. Training
  8. Experiments and Results
  9. Dataset
  10. EEG Preprocessing
  11. Speech Preprocessing
  12. Decoding Procedure
  13. DARNet
  14. Our Model
  15. Results
  16. ...and 9 more sections

Figures (5)

  • Figure 1: Comparison between the dichotic and diotic settings. (a) The dichotic setup presents distinct speech streams to each ear that enables decoding based on spatial location. (b) The diotic setup presents a mixture of $N$ speech streams to both ears. This requires the decoder to identify the attended speech without using speech source direction information.
  • Figure 2: Overview of the proposed diotic AAD framework.
  • Figure 3: Comparison of classification accuracy between DARNet and the proposed method across different segment lengths (1s, 3s, and 5s). The bar heights represent the mean accuracy obtained from 7-fold cross-validation, and the error bars indicate the standard deviation. The dashed gray line represents the chance level (50%).
  • Figure 4: Conceptual diagram of the M-MM task. The goal is to verify whether our proposed framework operates through solely acoustic matching or late-stage attention by comparing acoustic decoding accuracy between attended and unattended streams. EEG segment at time $t$ is paired with temporally matching speech segments and temporally mismatching segments (at different time $t'$). The model discriminates which speech segment temporally corresponds to the EEG, independently for attended and unattended streams, to evaluate speech decoding accuracy independent of attentional state.
  • Figure 5: Comparison of SHAP-based EEG spatial importance. Left: M-MM task (acoustic encoding) exhibits importance clusters in central-lateral regions (FC6, C3, FC5, T8). This reflects auditory sensory processing. Middle: Diotic AAD task (attentional selection) shows distinct importance of frontal (Fz, F3, F4) and temporo-parietal (TP9, TP10) regions. Right: The difference map highlights attention-specific frontal engagement and a relative decrease in central-lateral importance. This is consistent with the recruitment of top-down control networks.