A New Approach to Voice Authenticity

TL;DR

This work argues that a binary genuine/fake view of voice content is insufficient in the era of editing, synthesis, and manipulation. It introduces a $6$-category taxonomy of voice edits comprising $21$ distinct edits, backed by a new dataset framework built on the M-AILABS corpus to enable real-time augmentation and evaluation. Four neural architectures (LCNN, ComplexNet, RawNet2, SSL W2V2) are benchmarked across time resolutions ($0.35$ s, $1.2$ s, $4.05$ s), revealing that detection performance degrades with finer granularity, while pretrained models excel at coarse scales and specialized models perform best at intermediate scales. The study advocates shifting research focus from fake-vs-real to comprehensive detection of voice edits to better tackle misinformation, legal verification, and security in real-world audio content, and provides practical baselines and use-case analysis to guide future work.

Abstract

Voice faking, driven primarily by recent advances in text-to-speech (TTS) synthesis technology, poses significant societal challenges. Currently, the prevailing assumption is that unaltered human speech can be considered genuine, while fake speech comes from TTS synthesis. We argue that this binary distinction is oversimplified. For instance, altered playback speeds can be used for malicious purposes, like in the 'Drunken Nancy Pelosi' incident. Similarly, editing of audio clips can be done ethically, e.g., for brevity or summarization in news reporting or podcasts, but editing can also create misleading narratives. In this paper, we propose a conceptual shift away from the binary paradigm of audio being either 'fake' or 'real'. Instead, our focus is on pinpointing 'voice edits', which encompass traditional modifications like filters and cuts, as well as TTS synthesis and VC systems. We delineate 6 categories and curate a new challenge dataset rooted in the M-AILABS corpus, for which we present baseline detection systems. And most importantly, we argue that merely categorizing audio as fake or real is a dangerous over-simplification that will fail to move the field of speech technology forward.