Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Test-Time Adaptation For Speech Enhancement Via Mask Polarization

Tobias Raichle, Erfan Amini, Bin Yang

TL;DR

The paper tackles the problem of mask-based speech enhancement failing under unseen domain shifts. It introduces Mask Polarization (MPol), a lightweight test-time adaptation method that aligns the distribution of predicted masks to a bimodal reference mask using the Wasserstein distance, without adding model parameters. The approach combines a Wasserstein loss with a nonnegativity penalty and uses continual weight ensembling to stabilize updates, achieving very consistent gains across datasets and architectures and approaching the performance of heavier baselines like RemixIT and CMGAN. This work demonstrates that restoring mask bimodality is a robust, practical signal for adapting SE models in resource-constrained settings, bridging ideas from classification adaptation to audio tasks and enabling practical edge deployment.

Abstract

Adapting speech enhancement (SE) models to unseen environments is crucial for practical deployments, yet test-time adaptation (TTA) for SE remains largely under-explored due to a lack of understanding of how SE models degrade under domain shifts. We observe that mask-based SE models lose confidence under domain shifts, with predicted masks becoming flattened and losing decisive speech preservation and noise suppression. Based on this insight, we propose mask polarization (MPol), a lightweight TTA method that restores mask bimodality through distribution comparison using the Wasserstein distance. MPol requires no additional parameters beyond the trained model, making it suitable for resource-constrained edge deployments. Experimental results across diverse domain shifts and architectures demonstrate that MPol achieves very consistent gains that are competitive with significantly more complex approaches.

Test-Time Adaptation For Speech Enhancement Via Mask Polarization

TL;DR

The paper tackles the problem of mask-based speech enhancement failing under unseen domain shifts. It introduces Mask Polarization (MPol), a lightweight test-time adaptation method that aligns the distribution of predicted masks to a bimodal reference mask using the Wasserstein distance, without adding model parameters. The approach combines a Wasserstein loss with a nonnegativity penalty and uses continual weight ensembling to stabilize updates, achieving very consistent gains across datasets and architectures and approaching the performance of heavier baselines like RemixIT and CMGAN. This work demonstrates that restoring mask bimodality is a robust, practical signal for adapting SE models in resource-constrained settings, bridging ideas from classification adaptation to audio tasks and enabling practical edge deployment.

Abstract

Adapting speech enhancement (SE) models to unseen environments is crucial for practical deployments, yet test-time adaptation (TTA) for SE remains largely under-explored due to a lack of understanding of how SE models degrade under domain shifts. We observe that mask-based SE models lose confidence under domain shifts, with predicted masks becoming flattened and losing decisive speech preservation and noise suppression. Based on this insight, we propose mask polarization (MPol), a lightweight TTA method that restores mask bimodality through distribution comparison using the Wasserstein distance. MPol requires no additional parameters beyond the trained model, making it suitable for resource-constrained edge deployments. Experimental results across diverse domain shifts and architectures demonstrate that MPol achieves very consistent gains that are competitive with significantly more complex approaches.
Paper Structure (13 sections, 7 equations, 4 figures, 2 tables)

This paper contains 13 sections, 7 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Related Work
  4. Methodology
  5. Mask Polarization
  6. Experiments
  7. Datasets
  8. Models
  9. Experimental Details
  10. Results
  11. Results Analysis
  12. Ablation Study
  13. Conclusion

Figures (4)

  • Figure 1: Overview of mpol (mpol). The mask histogram flattens under domain shifts.
  • Figure 2: Comparison of masks and their histograms.
  • Figure 3: $\Delta$PESQ results relative to the source performance using the AM architecture ($\mu\pm2\sigma$).
  • Figure 4: $\Delta$PESQ results relative to the source performance using the CMGAN architecture ($\mu\pm2\sigma$).