EchoFake: A Replay-Aware Dataset for Practical Speech Deepfake Detection

TL;DR

This work addresses the vulnerability of speech deepfake detectors to practical replay attacks by introducing EchoFake, a replay-aware dataset that blends zero-shot TTS deepfakes with diverse physical replay recordings. The dataset spans four subsets and 11 TTS systems across 13k speakers, enabling realistic evaluation under multiple replay conditions. Experiments with three baselines show significant open-set degradation from replay, but training on EchoFake improves cross-dataset generalization as reflected in lower $EER$ on benchmarks. The dataset provides a realistic benchmark to push toward deployable, replay-robust spoofing detectors and is openly available for reproducible research.

Abstract

The growing prevalence of speech deepfakes has raised serious concerns, particularly in real-world scenarios such as telephone fraud and identity theft. While many anti-spoofing systems have demonstrated promising performance on lab-generated synthetic speech, they often fail when confronted with physical replay attacks-a common and low-cost form of attack used in practical settings. Our experiments show that models trained on existing datasets exhibit severe performance degradation, with average accuracy dropping to 59.6% when evaluated on replayed audio. To bridge this gap, we present EchoFake, a comprehensive dataset comprising more than 120 hours of audio from over 13,000 speakers, featuring both cutting-edge zero-shot text-to-speech (TTS) speech and physical replay recordings collected under varied devices and real-world environmental settings. Additionally, we evaluate three baseline detection models and show that models trained on EchoFake achieve lower average EERs across datasets, indicating better generalization. By introducing more practical challenges relevant to real-world deployment, EchoFake offers a more realistic foundation for advancing spoofing detection methods.

Figures (2)

• Figure 1: ADD models trained on ASVspoof dataset often fail in real-world conditions. (a) When ordinary users employ existing ADD models, the models are prone to misclassifying genuine speech as fake due to variations in recording environments and speaker characteristics. (b) Attackers can simply replay synthetic speech to bypass detection, causing the model to misclassify fake speech as genuine.
• Figure 2: Pipeline for constructing the EchoFake dataset. Abbreviations: B (bona fide), RB (replayed bona fide), F (fake), RF (replayed fake).