A Hybrid Discriminative and Generative System for Universal Speech Enhancement

TL;DR

This work introduces a hybrid universal speech enhancement system that fuses discriminative fidelity from TF-GridNet with a generative, semantically conditioned refinement to handle diverse distortions and variable sampling rates. A fusion network adaptively weighs the two outputs under joint signal-level and perceptual losses, including a multi-metric SQA objective, to suppress artifacts while preserving detail. Evaluations on URGENT 2026 Track 1 show the hybrid achieves competitive objective fidelity (PESQ, ESTOI) and superior perceptual quality (DNSMOS, NISQA), with strong speaker similarity and ASR accuracy, indicating the approach effectively leverages complementary strengths of discriminative and generative modeling. Limitations include the 16 kHz operating band and notable inference latency in the generative branch; future work aims at full-band processing and efficiency improvements for real-time deployment.

Abstract

Universal speech enhancement aims at handling inputs with various speech distortions and recording conditions. In this work, we propose a novel hybrid architecture that synergizes the signal fidelity of discriminative modeling with the reconstruction capabilities of generative modeling. Our system utilizes the discriminative TF-GridNet model with the Sampling-Frequency-Independent strategy to handle variable sampling rates universally. In parallel, an autoregressive model combined with spectral mapping modeling generates detail-rich speech while effectively suppressing generative artifacts. Finally, a fusion network learns adaptive weights of the two outputs under the optimization of signal-level losses and the comprehensive Speech Quality Assessment (SQA) loss. Our proposed system is evaluated in the ICASSP 2026 URGENT Challenge (Track 1) and ranks the third place.

Figures (1)

• Figure 1: Architecture of the proposed generative branch model.