Robust Channel Learning for Large-Scale Radio Speaker Verification

TL;DR

Problem: speaker verification performance degrades under radio channel conditions, especially bandwidth constraints and noise. Approach: Channel Robust Speaker Learning (CRSL) combines BandNoiseAugment for bandwidth-aware augmentation, a radio-corpus collection toolkit to simulate radio transmission, and an early-fine-tuning strategy to adapt shallow layers efficiently. Contributions: BandNoiseAugment reduces EER/minDCF on radio data with minimal overhead, early fine-tuning improves robustness by targeting early convolutional layers, and a scalable radio speech corpus and benchmark enable reproducible evaluation of radio-channel effects. Findings: experiments on VoxCeleb1/2 and CNCeleb show substantial gains over baselines and reveal distinct roles of bandwidth and channel factors in feature extraction.

Abstract

Recent research in speaker verification has increasingly focused on achieving robust and reliable recognition under challenging channel conditions and noisy environments. Identifying speakers in radio communications is particularly difficult due to inherent limitations such as constrained bandwidth and pervasive noise interference. To address this issue, we present a Channel Robust Speaker Learning (CRSL) framework that enhances the robustness of the current speaker verification pipeline, considering data source, data augmentation, and the efficiency of model transfer processes. Our framework introduces an augmentation module that mitigates bandwidth variations in radio speech datasets by manipulating the bandwidth of training inputs. It also addresses unknown noise by introducing noise within the manifold space. Additionally, we propose an efficient fine-tuning method that reduces the need for extensive additional training time and large amounts of data. Moreover, we develop a toolkit for assembling a large-scale radio speech corpus and establish a benchmark specifically tailored for radio scenario speaker verification studies. Experimental results demonstrate that our proposed methodology effectively enhances performance and mitigates degradation caused by radio transmission in speaker verification tasks. The code will be available on Github.

Figures (8)

• Figure 1: An overview of the Channel Robust Speaker Learning framework for speaker verification. Left: Corpus Collection for radio communication; Medium: BandNoiseAugment for audio corpus; Right: Fine-tuning for early-stage convolutional layers.
• Figure 2: This is the GRC graph of the pipeline of the overall radio transmission framework. Pipeline A uses HackRF One to transmit the audio signal in the air. Pipeline B uses the Channel Model in GNU Radio to simulate the transmission with multiprocessing.
• Figure 3: BandNoiseAugment Module. Bandwidth manipulation is applied to the waveform. SVD and noise injection are applied to Mel fbanks.
• Figure 4: Early fine-tuning for radio speaker classification. The fine-tuning policy is derived from comparing the statistical drift ($d_i$ in Equation.\ref{['eq8']}) of output features between clean and radio corpora.
• Figure 5: Comparison of waveforms from NBFM and WBFM with original audio. The loss of detail is evident in the NBFM radio audio. The blue waveform is the original audio. The orange waveform is radio audio.