Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Schrödinger Bridge Mamba for One-Step Speech Enhancement

Jing Yang, Sirui Wang, Chao Wu, Fan Fan

TL;DR

This paper addresses slow inference in Schrödinger Bridge-based speech enhancement by introducing Schrödinger Bridge Mamba (SBM), a framework that trains a Mamba backbone under the SB paradigm to enable one-step inference. By aligning SB's optimal transport path with Mamba's Markov-state-space dynamics, SBM distills SB transformations into the state-space, delivering superior denoising and dereverberation performance with a minimal inference burden. Empirically, SBM outperforms SB-NCSN++ baselines and predictive-mapping Mamba models on DNS and VoiceBank-Demand datasets, achieving best or near-best scores across perceptual, semantic, and speaker similarity metrics while maintaining favorable real-time factors. The results indicate a strong, generalizable synergy between SB and Mamba, suggesting potential applicability to a broad range of generative tasks beyond speech, including image and multimodal domains.

Abstract

We propose Schrödinger Bridge Mamba (SBM), a new concept of training-inference framework motivated by the inherent compatibility between Schrödinger Bridge (SB) training paradigm and selective state-space model Mamba. We exemplify the concept of SBM with an implementation for generative speech enhancement. Experiments on a joint denoising and dereverberation task using four benchmark datasets demonstrate that SBM, with only 1-step inference, outperforms strong baselines with 1-step or iterative inference and achieves the best real-time factor (RTF). Beyond speech enhancement, we discuss the integration of SB paradigm and selective state-space model architecture based on their underlying alignment, which indicates a promising direction for exploring new deep generative models potentially applicable to a broad range of generative tasks. Demo page: https://sbmse.github.io

Schrödinger Bridge Mamba for One-Step Speech Enhancement

TL;DR

This paper addresses slow inference in Schrödinger Bridge-based speech enhancement by introducing Schrödinger Bridge Mamba (SBM), a framework that trains a Mamba backbone under the SB paradigm to enable one-step inference. By aligning SB's optimal transport path with Mamba's Markov-state-space dynamics, SBM distills SB transformations into the state-space, delivering superior denoising and dereverberation performance with a minimal inference burden. Empirically, SBM outperforms SB-NCSN++ baselines and predictive-mapping Mamba models on DNS and VoiceBank-Demand datasets, achieving best or near-best scores across perceptual, semantic, and speaker similarity metrics while maintaining favorable real-time factors. The results indicate a strong, generalizable synergy between SB and Mamba, suggesting potential applicability to a broad range of generative tasks beyond speech, including image and multimodal domains.

Abstract

We propose Schrödinger Bridge Mamba (SBM), a new concept of training-inference framework motivated by the inherent compatibility between Schrödinger Bridge (SB) training paradigm and selective state-space model Mamba. We exemplify the concept of SBM with an implementation for generative speech enhancement. Experiments on a joint denoising and dereverberation task using four benchmark datasets demonstrate that SBM, with only 1-step inference, outperforms strong baselines with 1-step or iterative inference and achieves the best real-time factor (RTF). Beyond speech enhancement, we discuss the integration of SB paradigm and selective state-space model architecture based on their underlying alignment, which indicates a promising direction for exploring new deep generative models potentially applicable to a broad range of generative tasks. Demo page: https://sbmse.github.io
Paper Structure (11 sections, 2 equations, 1 figure, 3 tables)

This paper contains 11 sections, 2 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Schrödinger Bridge for Speech Enhancement
  4. Mamba-based Model for Speech Enhancement
  5. Schrödinger Bridge Mamba
  6. Experiemental Setup
  7. Implementation Details
  8. Datasets and Metrics
  9. Baseline Models
  10. Results and Discussion
  11. Conclusion

Figures (1)

  • Figure 1: Overview of different training and inference paradigms. (a) Typical SB for SE: Training leverages SB paradigm, and inference leverages an iterative SDE or ODE sampling. The same denoising objectives as in diffusion models can be applied, and the figure illustrates a data prediction loss. (b) Typical Mamba-based model for SE: Predictive mapping training and inference paradigm is usually employed. (c) The presented SBM for SE: Training leverages SB paradigm to train a Mamba-based backbone model, and inference resembles predictive mapping inference with an additional timestep embedding.