MAPSS: Manifold-based Assessment of Perceptual Source Separation

TL;DR

MAPSS presents Perceptual Separation (PS) and Perceptual Match (PM), two frame-level, differentiable metrics that disentangle leakage from self-distortion in source separation by embedding self-supervised representations and distortions onto a diffusion-map manifold. The method constructs perceptual clusters around each reference, computes Mahalanobis distances for leakage and match, and derives deterministic truncation bounds and non-asymptotic confidence intervals to quantify uncertainty. Empirical evaluation on SEBASS English, Spanish, and music mixtures shows PS and PM achieving top correlations with human MOS relative to 14 baselines, with complementary behavior as system quality degrades. The work provides both practical metrics for reliable evaluation and theoretical guarantees, and offers open-source tooling to foster perceptually grounded benchmarking in the community.

Abstract

Objective assessment of source-separation systems still mismatches subjective human perception, especially when leakage and self-distortion interact. We introduce the Perceptual Separation (PS) and Perceptual Match (PM), the first pair of measures that functionally isolate these two factors. Our intrusive method begins with generating a bank of fundamental distortions for each reference waveform signal in the mixture. Distortions, references, and their respective system outputs from all sources are then independently encoded by a pre-trained self-supervised learning model. These representations are aggregated and projected onto a manifold via diffusion maps, which aligns Euclidean distances on the manifold with dissimilarities of the encoded waveforms. On this manifold, the PM measures the Mahalanobis distance from each output to its attributed cluster that consists of its reference and distortions embeddings, capturing self-distortion. The PS accounts for the Mahalanobis distance of the output to the attributed and to the closest non-attributed clusters, quantifying leakage. Both measures are differentiable and granular, operating at a resolution as low as 50 frames per second. We further derive, for both measures, deterministic error radius and non-asymptotic, high-probability confidence intervals (CIs). Experiments on English, Spanish, and music mixtures show that the PS and PM nearly always achieve the highest linear correlation coefficients with human mean-opinion scores than 14 competitors, reaching as high as 86.36% for speech and 87.21% for music. We observe, at worst, an error radius of 1.39% and a probabilistic 95% CI of 12.21% for these coefficients, which improves reliable and informed evaluation. Using mutual information, the measures complement each other most as their values decrease, suggesting they are jointly more informative as system performance degrades.

Figures (10)

• Figure 1: NMI between the PS and PM measures across their thresholded values.
• Figure 2: Functional behavior of the PS measure with $0.25, 0.5, 0.75$ contour lines, and of the PM measure in three different setups of $\theta^{(d)}_{i}=1, a^{(d)}_{i}=1, k^{(d)}_{i}=1$.
• Figure 3: The distribution of PM and PS values across speech and music scenarios from the SEBASS database.
• Figure 4: Frame-level SNR estimations for English and Spanish references in the SEBASS database.
• Figure 5: For English mixtures, the effect of transformer layers in different pretrained self-supervised models on the PCC and SRCC values for the PS and PM measures.