Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Geometry-Constrained EEG Channel Selection for Brain-Assisted Speech Enhancement

Keying Zuo, Qingtian Xu, Jie Zhang, Zhenhua Ling

TL;DR

A new weighted multi-dilation temporal convolutional network (WD-TCN) as the backbone to replace the Conv-TasNet in BASEN and a geometry-constrained convolutional regularization selection (GC-ConvRS) module for WD-TCN to find an informative EEG subset subject to the geometry constraint.

Abstract

Brain-assisted speech enhancement (BASE) aims to extract the target speaker in complex multi-talker scenarios using electroencephalogram (EEG) signals as an assistive modality, as the auditory attention of the listener can be decoded from electroneurographic signals of the brain. This facilitates a potential integration of EEG electrodes with listening devices to improve the speech intelligibility of hearing-impaired listeners, which was shown by the recently-proposed BASEN model. As in general the multichannel EEG signals are highly correlated and some are even irrelevant to listening, blindly incorporating all EEG channels would lead to a high economic and computational cost. In this work, we therefore propose a geometry-constrained EEG channel selection approach for BASE. We design a new weighted multi-dilation temporal convolutional network (WDTCN) as the backbone to replace the Conv-TasNet in BASEN. Given a raw channel set that is defined by the electrode geometry for feasible integration, we then propose a geometry-constrained convolutional regularization selection (GC-ConvRS) module for WD-TCN to find an informative EEG subset. Experimental results on a public dataset show the superiority of the proposed WD-TCN over BASEN. The GC-ConvRS can further refine the useful EEG subset subject to the geometry constraint, resulting in a better trade-off between performance and integration cost.

Geometry-Constrained EEG Channel Selection for Brain-Assisted Speech Enhancement

TL;DR

A new weighted multi-dilation temporal convolutional network (WD-TCN) as the backbone to replace the Conv-TasNet in BASEN and a geometry-constrained convolutional regularization selection (GC-ConvRS) module for WD-TCN to find an informative EEG subset subject to the geometry constraint.

Abstract

Brain-assisted speech enhancement (BASE) aims to extract the target speaker in complex multi-talker scenarios using electroencephalogram (EEG) signals as an assistive modality, as the auditory attention of the listener can be decoded from electroneurographic signals of the brain. This facilitates a potential integration of EEG electrodes with listening devices to improve the speech intelligibility of hearing-impaired listeners, which was shown by the recently-proposed BASEN model. As in general the multichannel EEG signals are highly correlated and some are even irrelevant to listening, blindly incorporating all EEG channels would lead to a high economic and computational cost. In this work, we therefore propose a geometry-constrained EEG channel selection approach for BASE. We design a new weighted multi-dilation temporal convolutional network (WDTCN) as the backbone to replace the Conv-TasNet in BASEN. Given a raw channel set that is defined by the electrode geometry for feasible integration, we then propose a geometry-constrained convolutional regularization selection (GC-ConvRS) module for WD-TCN to find an informative EEG subset. Experimental results on a public dataset show the superiority of the proposed WD-TCN over BASEN. The GC-ConvRS can further refine the useful EEG subset subject to the geometry constraint, resulting in a better trade-off between performance and integration cost.
Paper Structure (8 sections, 6 equations, 3 figures)

This paper contains 8 sections, 6 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Methodology
  3. WD-TCN
  4. GC-ConvRS
  5. Performance Evaluation
  6. Experimental Setup
  7. Experimental Results
  8. Conclusion

Figures (3)

  • Figure 1: The proposed GC-ConvRS sparsity-driven brain-assisted speech enhancement method: (a) The backbone WD-TCN model, (b) WDDepthConv1D block, (c) GC-ConvRS, (d) SEAttention and (e) Loss function.
  • Figure 2: Performance of WD-TCN and BASEN zhang2023basen across subjects, where the median values are shown at the top of plots.
  • Figure 3: The performance of and visualization of GC-ConvRS, where the blue dots represent the selected electrodes.