BanglaRobustNet: A Hybrid Denoising-Attention Architecture for Robust Bangla Speech Recognition

TL;DR

BanglaRobustNet tackles robust Bangla ASR under noise and speaker variability by integrating a phonetic-aware diffusion-based denoiser with a speaker-conditioned contextual cross-attention mechanism atop a Wav2Vec-BERT backbone. The system is trained end-to-end with a multi-objective loss that jointly optimizes CTC transcription, phonetic fidelity, and speaker consistency, yielding substantial relative reductions in $WER$ and $CER$ compared to Whisper and Wav2Vec-BERT baselines across clean and noisy data. Key contributions include a diffusion-based denoising module that preserves Bangla phonemes and a cross-attention module conditioned on speaker embeddings (gender, age, dialect) to adapt to diverse speakers, enabling improved performance across dialects and demographics. The authors report strong improvements in noise-robustness and computational efficiency (real-time capable) and provide open-source code to advance research in low-resource languages and robust ASR deployment.

Abstract

Bangla, one of the most widely spoken languages, remains underrepresented in state-of-the-art automatic speech recognition (ASR) research, particularly under noisy and speaker-diverse conditions. This paper presents BanglaRobustNet, a hybrid denoising-attention framework built on Wav2Vec-BERT, designed to address these challenges. The architecture integrates a diffusion-based denoising module to suppress environmental noise while preserving Bangla-specific phonetic cues, and a contextual cross-attention module that conditions recognition on speaker embeddings for robustness across gender, age, and dialects. Trained end-to-end with a composite objective combining CTC loss, phonetic consistency, and speaker alignment, BanglaRobustNet achieves substantial reductions in word error rate (WER) and character error rate (CER) compared to Wav2Vec-BERT and Whisper baselines. Evaluations on Mozilla Common Voice Bangla and augmented noisy speech confirm the effectiveness of our approach, establishing BanglaRobustNet as a robust ASR system tailored to low-resource, noise-prone linguistic settings.

Figures (1)

• Figure 1: The proposed BanglaRobustNet architecture integrating denoising and attention mechanisms for robust Bangla speech recognition.