Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale

Xin Wang, Hector Delgado, Hemlata Tak, Jee-weon Jung, Hye-jin Shim, Massimiliano Todisco, Ivan Kukanov, Xuechen Liu, Md Sahidullah, Tomi Kinnunen, Nicholas Evans, Kong Aik Lee, Junichi Yamagishi

TL;DR

ASVspoof 5 tackles spoofing and deepfake threats to automatic speaker verification by introducing crowdsourced, diverse speech data and adversarial attacks, across two tracks (stand-alone CM and SASV). It provides a new MLS-based database and updated attack types, including adversarial variants, with an open data condition to broaden evaluation. The evaluation framework uses minDCF for Track 1 and a-DCF for Track 2, complemented by t-DCF and t-EER, enabling assessment of both detection and calibration. Results show substantial gains by top submissions over baselines even with non-studio data, and SSL-based features frequently improve performance, though score calibration remains a critical bottleneck for practical deployment.

Abstract

ASVspoof 5 is the fifth edition in a series of challenges that promote the study of speech spoofing and deepfake attacks, and the design of detection solutions. Compared to previous challenges, the ASVspoof 5 database is built from crowdsourced data collected from a vastly greater number of speakers in diverse acoustic conditions. Attacks, also crowdsourced, are generated and tested using surrogate detection models, while adversarial attacks are incorporated for the first time. New metrics support the evaluation of spoofing-robust automatic speaker verification (SASV) as well as stand-alone detection solutions, i.e., countermeasures without ASV. We describe the two challenge tracks, the new database, the evaluation metrics, baselines, and the evaluation platform, and present a summary of the results. Attacks significantly compromise the baseline systems, while submissions bring substantial improvements.

ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale

TL;DR

ASVspoof 5 tackles spoofing and deepfake threats to automatic speaker verification by introducing crowdsourced, diverse speech data and adversarial attacks, across two tracks (stand-alone CM and SASV). It provides a new MLS-based database and updated attack types, including adversarial variants, with an open data condition to broaden evaluation. The evaluation framework uses minDCF for Track 1 and a-DCF for Track 2, complemented by t-DCF and t-EER, enabling assessment of both detection and calibration. Results show substantial gains by top submissions over baselines even with non-studio data, and SSL-based features frequently improve performance, though score calibration remains a critical bottleneck for practical deployment.

Abstract

ASVspoof 5 is the fifth edition in a series of challenges that promote the study of speech spoofing and deepfake attacks, and the design of detection solutions. Compared to previous challenges, the ASVspoof 5 database is built from crowdsourced data collected from a vastly greater number of speakers in diverse acoustic conditions. Attacks, also crowdsourced, are generated and tested using surrogate detection models, while adversarial attacks are incorporated for the first time. New metrics support the evaluation of spoofing-robust automatic speaker verification (SASV) as well as stand-alone detection solutions, i.e., countermeasures without ASV. We describe the two challenge tracks, the new database, the evaluation metrics, baselines, and the evaluation platform, and present a summary of the results. Attacks significantly compromise the baseline systems, while submissions bring substantial improvements.
Paper Structure (15 sections, 5 equations, 2 figures, 5 tables)

This paper contains 15 sections, 5 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Database
  3. Performance measures
  4. Track 1: from EER to DCF
  5. Track 2: from SASV-EER to a-DCF
  6. Common ASV, surrogate systems, and challenge baselines
  7. Common ASV system by organisers
  8. Baseline systems
  9. Surrogate systems
  10. Evaluation platform
  11. Challenge results
  12. Track 1
  13. Track 2
  14. Conclusions
  15. Acknowledgements

Figures (2)

  • Figure 1: ASV EER of organiser's common ASV on evaluation data. Results are pooled over all codec conditions.
  • Figure 2: A stacked bar chart showing the number of submissions to Track 1 and 2 for the Codalab progress and evaluation phases (last three days).