A robust audio deepfake detection system via multi-view feature

TL;DR

This work addresses the generalization gap in audio deepfake detection by evaluating a wide range of handcrafted and learning-based audio features and demonstrating that large-scale pretraining yields superior cross-domain performance. The authors introduce two multi-view feature incorporation methods—sample-aware feature selection and feature fusion with channel attention and a Transformer encoder—to harness complementary information from multiple features. Experiments on ASVSpoof and In-the-Wild show that multi-view fusion reduces the equal-error rate to 24.27% in realistic conditions, illustrating improved robustness for real-world ADD. The findings underscore the importance of leveraging diverse, pretraining-rich audio representations and effective multi-view integration to enhance ADD performance in challenging, real-world environments.

Abstract

With the advancement of generative modeling techniques, synthetic human speech becomes increasingly indistinguishable from real, and tricky challenges are elicited for the audio deepfake detection (ADD) system. In this paper, we exploit audio features to improve the generalizability of ADD systems. Investigation of the ADD task performance is conducted over a broad range of audio features, including various handcrafted features and learning-based features. Experiments show that learning-based audio features pretrained on a large amount of data generalize better than hand-crafted features on out-of-domain scenarios. Subsequently, we further improve the generalizability of the ADD system using proposed multi-feature approaches to incorporate complimentary information from features of different views. The model trained on ASV2019 data achieves an equal error rate of 24.27\% on the In-the-Wild dataset.

Figures (1)

• Figure 1: Visualization of CQT features and Hubert features for real and fake speech in the In-the-Wild dataset.