Samba-ASR: State-Of-The-Art Speech Recognition Leveraging Structured State-Space Models

TL;DR

Samba-ASR introduces a transformer-free Automatic Speech Recognition system built on Mamba State Space Models that run with linear-time complexity, addressing the quadratic scaling challenge of transformers for long audio sequences. The architecture uses Mamba blocks for both the audio encoder and the text decoder, connected via a Mamba-cross-connection to align acoustic features with generated text, enabling efficient long-range dependency modeling. Empirical results on GigaSpeech, LibriSpeech, and SPGISpeech demonstrate state-of-the-art Word Error Rates, with notable performance on LibriSpeech Clean ($WER \approx 1.17\%$) and SPGISpeech ($WER \approx 1.84\%$), and competitive results on more challenging splits such as GigaSpeech ($WER \approx 9.12\%$). The approach achieves these gains with linear scaling and reduced latency, making Samba-ASR a scalable, robust alternative to transformer-based ASR while maintaining high accuracy across diverse datasets and noise conditions. This work highlights the viability of Mamba SSMs as an efficient backbone for ASR, potentially enabling broader deployment in real-time and resource-constrained settings.

Abstract

We propose Samba ASR,the first state of the art Automatic Speech Recognition(ASR)model leveraging the novel Mamba architecture as both encoder and decoder,built on the foundation of state space models(SSMs).Unlike transformerbased ASR models,which rely on self-attention mechanisms to capture dependencies,Samba ASR effectively models both local and global temporal dependencies using efficient statespace dynamics,achieving remarkable performance gains.By addressing the limitations of transformers,such as quadratic scaling with input length and difficulty in handling longrange dependencies,Samba ASR achieves superior accuracy and efficiency.Experimental results demonstrate that Samba ASR surpasses existing opensource transformerbased ASR models across various standard benchmarks,establishing it as the new state of theart in ASR.Extensive evaluations on the benchmark dataset show significant improvements in Word Error Rate(WER),with competitive performance even in lowresource scenarios.Furthermore,the inherent computational efficiency and parameter optimization of the Mamba architecture make Samba ASR a scalable and robust solution for diverse ASR tasks.Our contributions include the development of a new Samba ASR architecture for automatic speech recognition(ASR),demonstrating the superiority of structured statespace models(SSMs)over transformer based models for speech sequence processing.We provide a comprehensive evaluation on public benchmarks,showcasing stateoftheart(SOTA)performance,and present an indepth analysis of computational efficiency,robustness to noise,and sequence generalization.This work highlights the viability of Mamba SSMs as a transformerfree alternative for efficient and accurate ASR.By leveraging the advancements of statespace modeling,Samba ASR redefines ASR performance standards and sets a new benchmark for future research in this field.

Figures (3)

• Figure 1: Architecture diagram (original) of the Samba-ASR model, illustrating the key components including the Mamba encoder, which processes raw audio features using Mamba blocks, and the Mamba decoder along with the Mamba-Cross-Connection bridge, which generates transcriptions by integrating audio context with text representations. The model's design focuses on efficient long-range dependency capture for accurate automatic speech recognition.
• Figure 2: This graph shows the correlation of training and validation loss across epochs, with both losses steadily decreasing and converging around the 72nd epoch.
• Figure 3: This graph demonstrates a significant reduction in Word Error Rate (WER) throughout the training process, indicating improved model performance and accuracy.