Neural Codec Source Tracing: Toward Comprehensive Attribution in Open-Set Condition

TL;DR

This work defines Neural Codec Source Tracing (NCST) to enable open-set neural codec attribution and ALM-driven out-of-distribution detection in audio. It introduces the ST-Codecfake dataset, containing bilingual audio samples generated by 11 neural codecs and ALM-based OOD test samples, along with a comprehensive open-set source tracing benchmark. Experimental results show strong performance in in-distribution classification and OOD detection, but limited robustness when classifying unseen real audio. The dataset and code are publicly available to support reproducibility and drive further research in robust open-set attribution for audio deepfake sources.

Abstract

Current research in audio deepfake detection is gradually transitioning from binary classification to multi-class tasks, referred as audio deepfake source tracing task. However, existing studies on source tracing consider only closed-set scenarios and have not considered the challenges posed by open-set conditions. In this paper, we define the Neural Codec Source Tracing (NCST) task, which is capable of performing open-set neural codec classification and interpretable ALM detection. Specifically, we constructed the ST-Codecfake dataset for the NCST task, which includes bilingual audio samples generated by 11 state-of-the-art neural codec methods and ALM-based out-ofdistribution (OOD) test samples. Furthermore, we establish a comprehensive source tracing benchmark to assess NCST models in open-set conditions. The experimental results reveal that although the NCST models perform well in in-distribution (ID) classification and OOD detection, they lack robustness in classifying unseen real audio. The ST-codecfake dataset and code are available.

Figures (1)

• Figure 1: Schematic diagram of speech production.