Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ASoBO: Attentive Beamformer Selection for Distant Speaker Diarization in Meetings

Theo Mariotte, Anthony Larcher, Silvio Montresor, Jean-Hugh Thomas

TL;DR

ASoBO introduces a front-end for distant speaker diarization that replaces explicit DoA estimation with a fixed bank of directional beamformers whose outputs are adaptively reweighted by a Self-Attention Channel Combinator to produce a robust VAD+OSD feature sequence. This feature sequence feeds a VBx-based diarization pipeline, achieving competitive DER on AMI and AISHELL-4, notably 14.5% on AISHELL-4. The self-attention weights provide pseudo-localization of speakers, offering interpretability of the model's decisions. By avoiding explicit DoA estimation, ASoBO offers a compact, interpretable solution for distant-meeting diarization with strong practical potential.

Abstract

Speaker Diarization (SD) aims at grouping speech segments that belong to the same speaker. This task is required in many speech-processing applications, such as rich meeting transcription. In this context, distant microphone arrays usually capture the audio signal. Beamforming, i.e., spatial filtering, is a common practice to process multi-microphone audio data. However, it often requires an explicit localization of the active source to steer the filter. This paper proposes a self-attention-based algorithm to select the output of a bank of fixed spatial filters. This method serves as a feature extractor for joint Voice Activity (VAD) and Overlapped Speech Detection (OSD). The speaker diarization is then inferred from the detected segments. The approach shows convincing distant VAD, OSD, and SD performance, e.g. 14.5% DER on the AISHELL-4 dataset. The analysis of the self-attention weights demonstrates their explainability, as they correlate with the speaker's angular locations.

ASoBO: Attentive Beamformer Selection for Distant Speaker Diarization in Meetings

TL;DR

ASoBO introduces a front-end for distant speaker diarization that replaces explicit DoA estimation with a fixed bank of directional beamformers whose outputs are adaptively reweighted by a Self-Attention Channel Combinator to produce a robust VAD+OSD feature sequence. This feature sequence feeds a VBx-based diarization pipeline, achieving competitive DER on AMI and AISHELL-4, notably 14.5% on AISHELL-4. The self-attention weights provide pseudo-localization of speakers, offering interpretability of the model's decisions. By avoiding explicit DoA estimation, ASoBO offers a compact, interpretable solution for distant-meeting diarization with strong practical potential.

Abstract

Speaker Diarization (SD) aims at grouping speech segments that belong to the same speaker. This task is required in many speech-processing applications, such as rich meeting transcription. In this context, distant microphone arrays usually capture the audio signal. Beamforming, i.e., spatial filtering, is a common practice to process multi-microphone audio data. However, it often requires an explicit localization of the active source to steer the filter. This paper proposes a self-attention-based algorithm to select the output of a bank of fixed spatial filters. This method serves as a feature extractor for joint Voice Activity (VAD) and Overlapped Speech Detection (OSD). The speaker diarization is then inferred from the detected segments. The approach shows convincing distant VAD, OSD, and SD performance, e.g. 14.5% DER on the AISHELL-4 dataset. The analysis of the self-attention weights demonstrates their explainability, as they correlate with the speaker's angular locations.
Paper Structure (22 sections, 4 equations, 2 figures, 3 tables)

This paper contains 22 sections, 4 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Segmentation for speaker diarization
  3. Feature extraction
  4. Frame classification
  5. Integrating segmentation for speaker diarization
  6. Self-Attentive beamformer selection
  7. Super-directive beamforming
  8. Beamformer selection
  9. Experimental protocol
  10. Datasets
  11. Implementation details
  12. Baselines
  13. Training and evaluation
  14. Experimental study
  15. Segmentation performance
  16. ...and 7 more sections

Figures (2)

  • Figure 1: High-level architecture diagram of the proposed ASoBO and the speaker diarization pipeline. The $\theta_p$ blocks represent the fixed spatial filters. Only dash line blocks are trained.
  • Figure 2: (left) Combination weights for a spatialized utterance of Libri2Mix with speakers located at 0$^\circ$ and $90^\circ$. (right) Time-averaged weights from the same utterance.