Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decoding Speech Envelopes from Electroencephalogram with a Contrastive Pearson Correlation Coefficient Loss

Yayun Liang, Yuanming Zhang, Fei Chen, Jing Lu, Zhibin Lin

TL;DR

This work addresses EEG-based auditory attention decoding by reconstructing speech envelopes and introduces a contrastive Pearson correlation loss $ abla_{ ext{ΔPCC}}$ to explicitly maximize the difference between attended and unattended PCCs. The authors evaluate four diverse architectures across three public datasets, finding that the contrastive loss improves decoding accuracy and yields larger attended–unattended PCC gaps, with a notable average gain of about $17.84\%$. They observe that decoding performance correlates more strongly with $ abla_{ ext{ΔPCC}}$ than with $ ho_a$ alone, though stability varies by dataset and model, underscoring the need for robust optimization. Overall, the proposed objective offers a simple yet effective enhancement for EEG-based AAD with practical implications for multi-speaker brain–computer interfaces and attention-driven signal processing.

Abstract

Recent advances in reconstructing speech envelopes from Electroencephalogram (EEG) signals have enabled continuous auditory attention decoding (AAD) in multi-speaker environments. Most Deep Neural Network (DNN)-based envelope reconstruction models are trained to maximize the Pearson correlation coefficients (PCC) between the attended envelope and the reconstructed envelope (attended PCC). While the difference between the attended PCC and the unattended PCC plays an essential role in auditory attention decoding, existing methods often focus on maximizing the attended PCC. We therefore propose a contrastive PCC loss which represents the difference between the attended PCC and the unattended PCC. The proposed approach is evaluated on three public EEG AAD datasets using four DNN architectures. Across many settings, the proposed objective improves envelope separability and AAD accuracy, while also revealing dataset- and architecture-dependent failure cases.

Decoding Speech Envelopes from Electroencephalogram with a Contrastive Pearson Correlation Coefficient Loss

TL;DR

This work addresses EEG-based auditory attention decoding by reconstructing speech envelopes and introduces a contrastive Pearson correlation loss to explicitly maximize the difference between attended and unattended PCCs. The authors evaluate four diverse architectures across three public datasets, finding that the contrastive loss improves decoding accuracy and yields larger attended–unattended PCC gaps, with a notable average gain of about . They observe that decoding performance correlates more strongly with than with alone, though stability varies by dataset and model, underscoring the need for robust optimization. Overall, the proposed objective offers a simple yet effective enhancement for EEG-based AAD with practical implications for multi-speaker brain–computer interfaces and attention-driven signal processing.

Abstract

Recent advances in reconstructing speech envelopes from Electroencephalogram (EEG) signals have enabled continuous auditory attention decoding (AAD) in multi-speaker environments. Most Deep Neural Network (DNN)-based envelope reconstruction models are trained to maximize the Pearson correlation coefficients (PCC) between the attended envelope and the reconstructed envelope (attended PCC). While the difference between the attended PCC and the unattended PCC plays an essential role in auditory attention decoding, existing methods often focus on maximizing the attended PCC. We therefore propose a contrastive PCC loss which represents the difference between the attended PCC and the unattended PCC. The proposed approach is evaluated on three public EEG AAD datasets using four DNN architectures. Across many settings, the proposed objective improves envelope separability and AAD accuracy, while also revealing dataset- and architecture-dependent failure cases.
Paper Structure (17 sections, 6 equations, 1 figure, 1 table)

This paper contains 17 sections, 6 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Task description
  4. Model Architectures
  5. CNN-based Models
  6. Transformer-like models
  7. Loss Function
  8. Experiment
  9. Dataset
  10. Data Preprocessing
  11. Training Procedure
  12. Results and Discussion
  13. Correlation Between Accuracy and PCC
  14. Impact of Different Loss Functions on ACC and PCC Difference
  15. Mechanism of the proposed approach
  16. ...and 2 more sections

Figures (1)

  • Figure 1: The relationships between decoding accuracy (ACC) and (a) attended PCC, (b) PCC difference between the attended PCC and the unattended PCC ($\Delta\mathrm{PCC}$). Each point represents the decoding accuracy of a DNN decoder trained on a specific dataset using either $\mathcal{L}_{\mathrm{PCC}}$ or $\mathcal{L}_{\Delta\mathrm{PCC}}$. Each line represents a linear trend fitted on results of a DNN decoder.