Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mamba-SEUNet: Mamba UNet for Monaural Speech Enhancement

Junyu Wang, Zizhen Lin, Tianrui Wang, Meng Ge, Longbiao Wang, Jianwu Dang

TL;DR

Speech enhancement methods based on transformers face deployment challenges due to quadratic self-attention complexity. This work introduces Mamba-SEUNet, a U-Net–style SE network that embeds bidirectional Mamba blocks (a selective state-space model) and Taylor-based patch embeddings to capture long-range temporal and spectral dependencies across multiple resolutions. Key contributions include the TS-Mamba block design with bidirectional processing, an encoder–decoder architecture leveraging a Dilated DenseNet, and thorough evaluation on VCTK+DEMAND showing PESQ scores of $3.59$ (and $3.73$ with PCS), along with favorable FLOPs. The results demonstrate state-of-the-art SE performance with substantially lower computational demands, enabling more efficient deployment in real-time or resource-constrained scenarios.

Abstract

In recent speech enhancement (SE) research, transformer and its variants have emerged as the predominant methodologies. However, the quadratic complexity of the self-attention mechanism imposes certain limitations on practical deployment. Mamba, as a novel state-space model (SSM), has gained widespread application in natural language processing and computer vision due to its strong capabilities in modeling long sequences and relatively low computational complexity. In this work, we introduce Mamba-SEUNet, an innovative architecture that integrates Mamba with U-Net for SE tasks. By leveraging bidirectional Mamba to model forward and backward dependencies of speech signals at different resolutions, and incorporating skip connections to capture multi-scale information, our approach achieves state-of-the-art (SOTA) performance. Experimental results on the VCTK+DEMAND dataset indicate that Mamba-SEUNet attains a PESQ score of 3.59, while maintaining low computational complexity. When combined with the Perceptual Contrast Stretching technique, Mamba-SEUNet further improves the PESQ score to 3.73.

Mamba-SEUNet: Mamba UNet for Monaural Speech Enhancement

TL;DR

Speech enhancement methods based on transformers face deployment challenges due to quadratic self-attention complexity. This work introduces Mamba-SEUNet, a U-Net–style SE network that embeds bidirectional Mamba blocks (a selective state-space model) and Taylor-based patch embeddings to capture long-range temporal and spectral dependencies across multiple resolutions. Key contributions include the TS-Mamba block design with bidirectional processing, an encoder–decoder architecture leveraging a Dilated DenseNet, and thorough evaluation on VCTK+DEMAND showing PESQ scores of (and with PCS), along with favorable FLOPs. The results demonstrate state-of-the-art SE performance with substantially lower computational demands, enabling more efficient deployment in real-time or resource-constrained scenarios.

Abstract

In recent speech enhancement (SE) research, transformer and its variants have emerged as the predominant methodologies. However, the quadratic complexity of the self-attention mechanism imposes certain limitations on practical deployment. Mamba, as a novel state-space model (SSM), has gained widespread application in natural language processing and computer vision due to its strong capabilities in modeling long sequences and relatively low computational complexity. In this work, we introduce Mamba-SEUNet, an innovative architecture that integrates Mamba with U-Net for SE tasks. By leveraging bidirectional Mamba to model forward and backward dependencies of speech signals at different resolutions, and incorporating skip connections to capture multi-scale information, our approach achieves state-of-the-art (SOTA) performance. Experimental results on the VCTK+DEMAND dataset indicate that Mamba-SEUNet attains a PESQ score of 3.59, while maintaining low computational complexity. When combined with the Perceptual Contrast Stretching technique, Mamba-SEUNet further improves the PESQ score to 3.73.

Paper Structure

This paper contains 15 sections, 6 equations, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Mamba: Selective State Space Model
  3. Methodology
  4. Architecture Overview
  5. TS-Mamba Block
  6. Encoder and Decoder
  7. Experiments
  8. Datasets
  9. Experimental setup
  10. Evaluation metrics
  11. Results and analysis
  12. Performance Comparison with Baselines
  13. Ablation Study
  14. Mamba/Transformer/Conformer Block
  15. Conclusions

Figures (1)

  • Figure 1: The architecture of the proposed Mamba-SEUNet.