Can all variations within the unified mask-based beamformer framework achieve identical peak extraction performance?

TL;DR

This work presents a unified two-process framework for mask-based beamformers (BFs) aimed at target sound extraction. By cataloging 12 BF variations via operator and covariance-matrix combinations and introducing a BF-independent mask-based scaling step, the study demonstrates that all variations can achieve the theoretical upper-bound extraction performance on CHiME-4 data, under both individual and joint optimization. The results also show that mask-based scaling with L1-MN or L2-MN masks can emulate ideal scaling (IS), and that a practical parameter-count perspective explains why different BF formulations converge to the same peak performance. These findings provide a principled basis for designing TSE systems, improving scaling accuracy, and estimating BF performance in realistic settings, while also extending insights to ICA-based TSE methods due to shared formulations. The work further suggests that nonlinear post-processing may be required to surpass the upper-bound baseline in practice.

Abstract

This study investigates mask-based beamformers (BFs), which estimate filters for target sound extraction (TSE) using time-frequency masks. Although multiple mask-based BFs have been proposed, no consensus has been reached on which one offers the best target-extraction performance. Previously, we found that maximum signal-to-noise ratio and minimum mean square error (MSE) BFs can achieve the same extraction performance as the theoretical upper-bound performance, with each BF containing a different optimal mask. However, two issues remained unsolved: only two BFs were covered, excluding the minimum variance distortionless response BF, and ideal scaling (IS) was employed to ideally adjust the output scale, which is not applicable to realistic scenarios. To address these issues, this study proposes a unified framework for mask-based BFs comprising two processes: filter estimation that can cover all possible BFs and scaling applicable to realistic scenarios by employing a mask to generate a scaling reference. Based on the operators and covariance matrices used in BF formulas, all possible BFs can be classified into 12 variations, including two new ones. Optimal masks for both processes are obtained by minimizing the MSE between the target and BF output. The experimental results using the CHiME-4 dataset suggested that 1) all 12 variations can achieve the theoretical upper-bound performance, and 2) mask-based scaling can behave like IS, even when constraining the temporal mean of a non-negative mask to one. These results can be explained by considering the practical parameter count of the masks. These findings contribute to 1) designing a TSE system, 2) improving scaling accuracy through mask-based scaling, and 3) estimating the extraction performance of a BF.

Figures (12)

• Figure 1: Conceptual plot of the relationship between the closeness of the BF output to the target and mask values; the optimal mask denotes a set of mask values that achieve the BF output closest to the target.
• Figure 2: Mask type categorization based on the mask value constraint; the non-negative mask can be more constrained in two ways. One is a ratio mask and the other is a family of mean-normalized (MN) masks.
• Figure 3: Unified framework of mask-based BFs (proposed); this consists of two mask-based processes: filter estimation and scaling. This study assumes that the optimal masks for both processes are generated with virtual estimators.
• Figure 4: Process of generating observation data for three noisy scenarios; the multiplier $g$ was applied to the background (BG) noises before being mixed with the clean speeches to generate different noisy scenarios shown in Table \ref{['tab:snr-dev-set']}.
• Figure 5: System used in Experiments 1 and 2; for filter estimation, one or two mask buffers were provided depending on the variation used. Buffered mask values were iteratively updated by backpropagation (BP) to minimize the MSE loss. The effect of batch normalization (BN) was examined.