Speech Quality-Based Localization of Low-Quality Speech and Text-to-Speech Synthesis Artefacts
Michael Kuhlmann, Alexander Werning, Thilo von Neumann, Reinhold Haeb-Umbach
TL;DR
This paper tackles the interpretability gap in automatic speech quality assessment by introducing a consistency-based regularization that stabilizes frame-level quality predictions. By enforcing embedding and frame-level score consistency between full-context and sliced-context inputs, the approach reduces frame-level volatility while preserving utterance-level accuracy. The method is validated on partial spoof detection and artefact localization in state-of-the-art TTS systems, with listening tests confirming that low frame-level scores correlate with human-perceived poor quality in targeted segments. The results demonstrate improved localisation precision and offer a path toward more actionable, segment-level diagnostics for speech synthesis evaluation.
Abstract
A large number of works view the automatic assessment of speech from an utterance- or system-level perspective. While such approaches are good in judging overall quality, they cannot adequately explain why a certain score was assigned to an utterance. frame-level scores can provide better interpretability, but models predicting them are harder to tune and regularize since no strong targets are available during training. In this work, we show that utterance-level speech quality predictors can be regularized with a segment-based consistency constraint which notably reduces frame-level stochasticity. We then demonstrate two applications involving frame-level scores: The partial spoof scenario and the detection of synthesis artefacts in two state-of-the-art text-to-speech systems. For the latter, we perform listening tests and confirm that listeners rate segments to be of poor quality more often in the set defined by low frame-level scores than in a random control set.