Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MR-RawNet: Speaker verification system with multiple temporal resolutions for variable duration utterances using raw waveforms

Seung-bin Kim, Chan-yeong Lim, Jungwoo Heo, Ju-ho Kim, Hyun-seo Shin, Kyo-Won Koo, Ha-Jin Yu

TL;DR

MR-RawNet tackles speaker verification under variable-length utterances by introducing a dual-module approach that operates directly on raw waveforms. The Multi-Resolution Feature Extractor (MRFE) captures time-frequency information across multiple temporal scales, while the Multi-Resolution Attention (MRA) block aggregates diverse temporal contexts through parallel resolutions and attention-guided fusion. Together, MRFE and MRA improve robustness to utterance length, achieving state-of-the-art performance among raw-waveform SV systems on VoxCeleb datasets and outperforming RawNet3, particularly for short utterances. This approach enables more reliable speaker verification in real-world settings where speech duration varies, with practical impact in security and biometric applications.

Abstract

In speaker verification systems, the utilization of short utterances presents a persistent challenge, leading to performance degradation primarily due to insufficient phonetic information to characterize the speakers. To overcome this obstacle, we propose a novel structure, MR-RawNet, designed to enhance the robustness of speaker verification systems against variable duration utterances using raw waveforms. The MR-RawNet extracts time-frequency representations from raw waveforms via a multi-resolution feature extractor that optimally adjusts both temporal and spectral resolutions simultaneously. Furthermore, we apply a multi-resolution attention block that focuses on diverse and extensive temporal contexts, ensuring robustness against changes in utterance length. The experimental results, conducted on VoxCeleb1 dataset, demonstrate that the MR-RawNet exhibits superior performance in handling utterances of variable duration compared to other raw waveform-based systems.

MR-RawNet: Speaker verification system with multiple temporal resolutions for variable duration utterances using raw waveforms

TL;DR

MR-RawNet tackles speaker verification under variable-length utterances by introducing a dual-module approach that operates directly on raw waveforms. The Multi-Resolution Feature Extractor (MRFE) captures time-frequency information across multiple temporal scales, while the Multi-Resolution Attention (MRA) block aggregates diverse temporal contexts through parallel resolutions and attention-guided fusion. Together, MRFE and MRA improve robustness to utterance length, achieving state-of-the-art performance among raw-waveform SV systems on VoxCeleb datasets and outperforming RawNet3, particularly for short utterances. This approach enables more reliable speaker verification in real-world settings where speech duration varies, with practical impact in security and biometric applications.

Abstract

In speaker verification systems, the utilization of short utterances presents a persistent challenge, leading to performance degradation primarily due to insufficient phonetic information to characterize the speakers. To overcome this obstacle, we propose a novel structure, MR-RawNet, designed to enhance the robustness of speaker verification systems against variable duration utterances using raw waveforms. The MR-RawNet extracts time-frequency representations from raw waveforms via a multi-resolution feature extractor that optimally adjusts both temporal and spectral resolutions simultaneously. Furthermore, we apply a multi-resolution attention block that focuses on diverse and extensive temporal contexts, ensuring robustness against changes in utterance length. The experimental results, conducted on VoxCeleb1 dataset, demonstrate that the MR-RawNet exhibits superior performance in handling utterances of variable duration compared to other raw waveform-based systems.
Paper Structure (11 sections, 3 equations, 3 figures, 3 tables)

This paper contains 11 sections, 3 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Baseline
  3. Proposed methods
  4. Multi-resolution feature extractor
  5. Multi-resolution attention (MRA) block
  6. Experimental setup
  7. Datasets
  8. Configurations
  9. Results
  10. Conclusion
  11. Acknowledgement

Figures (3)

  • Figure 1: (a): Baseline structure. The kernel size of Conv1D is 1, and $p$ represents max pooling size. (b): MR-RawNet structure. $k$, $d$, $N$, and $B$ denote kernel size, dilation, the number of feature extractors, and the number of MRA blocks.
  • Figure 2: MRFE structure. $k$, $s$, $R$ and $X$ denote kernel size, stride size, the number of repeats, and the number of convolutional blocks in each repeat, respectively.
  • Figure 3: MRA structure. DownSample and UpSample refer to the transposed convolution and average pooling, respectively. The scale dimension of Res2Dilated Conv1D is 4.