Investigating Training Objectives for Generative Speech Enhancement

TL;DR

This work compares score-based generative models (SGMSE) and Schrödinger bridge (SB) approaches for generative speech enhancement, showing that these frameworks, while theoretically related, exhibit different training dynamics and practical behaviors. It introduces a perceptual loss for SB to improve perceptual quality and evaluates both families on the VB-DMD benchmark, demonstrating that SB with data-prediction and perceptual loss can achieve state-of-the-art PESQ while balancing other metrics. The study reveals that objective choices (e.g., loss type, perceptual terms, and samplers) materially affect performance and stability, offering guidance for deploying diffusion-based speech restoration systems. All experimental code and pre-trained models are released to facilitate ongoing research and reproducibility.

Abstract

Generative speech enhancement has recently shown promising advancements in improving speech quality in noisy environments. Multiple diffusion-based frameworks exist, each employing distinct training objectives and learning techniques. This paper aims to explain the differences between these frameworks by focusing our investigation on score-based generative models and the Schrödinger bridge. We conduct a series of comprehensive experiments to compare their performance and highlight differing training behaviors. Furthermore, we propose a novel perceptual loss function tailored for the Schrödinger bridge framework, demonstrating enhanced performance and improved perceptual quality of the enhanced speech signals. All experimental code and pre-trained models are publicly available to facilitate further research and development in this domain.

Figures (1)

• Figure 1: PESQ and SI-SDR performance over the training steps.