Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adversarial Speaker Distillation for Countermeasure Model on Automatic Speaker Verification

Yen-Lun Liao, Xuanjun Chen, Chung-Che Wang, Jyh-Shing Roger Jang

TL;DR

The paper tackles the challenge of deploying effective spoofing countermeasures (CM) for automatic speaker verification on edge devices with limited resources. It introduces adversarial speaker distillation, combining GE2E pre-training, adversarial fine-tuning, and knowledge distillation from a strong teacher to a compact student (ASD-ResNetSE). Key results show substantial gains from GE2E pre-training and adversarial augmentation, with the ASD-ResNetSE achieving a min t-DCF of 0.2695 and EER of 3.54% using only 22.5% of the ResNetSE parameters and 19.4% of its MACs. This approach demonstrates a practical path to edge-ready CM for ASV that balances performance and resource efficiency, outperforming many comparable methods while dramatically reducing model size and compute.

Abstract

The countermeasure (CM) model is developed to protect ASV systems from spoof attacks and prevent resulting personal information leakage in Automatic Speaker Verification (ASV) system. Based on practicality and security considerations, the CM model is usually deployed on edge devices, which have more limited computing resources and storage space than cloud-based systems, confining the model size under a limitation. To better trade off the CM model sizes and performance, we proposed an adversarial speaker distillation method, which is an improved version of knowledge distillation method combined with generalized end-to-end (GE2E) pre-training and adversarial fine-tuning. In the evaluation phase of the ASVspoof 2021 Logical Access task, our proposed adversarial speaker distillation ResNetSE (ASD-ResNetSE) model reaches 0.2695 min t-DCF and 3.54% EER. ASD-ResNetSE only used 22.5% of parameters and 19.4% of multiply and accumulate operands of ResNetSE model.

Adversarial Speaker Distillation for Countermeasure Model on Automatic Speaker Verification

TL;DR

The paper tackles the challenge of deploying effective spoofing countermeasures (CM) for automatic speaker verification on edge devices with limited resources. It introduces adversarial speaker distillation, combining GE2E pre-training, adversarial fine-tuning, and knowledge distillation from a strong teacher to a compact student (ASD-ResNetSE). Key results show substantial gains from GE2E pre-training and adversarial augmentation, with the ASD-ResNetSE achieving a min t-DCF of 0.2695 and EER of 3.54% using only 22.5% of the ResNetSE parameters and 19.4% of its MACs. This approach demonstrates a practical path to edge-ready CM for ASV that balances performance and resource efficiency, outperforming many comparable methods while dramatically reducing model size and compute.

Abstract

The countermeasure (CM) model is developed to protect ASV systems from spoof attacks and prevent resulting personal information leakage in Automatic Speaker Verification (ASV) system. Based on practicality and security considerations, the CM model is usually deployed on edge devices, which have more limited computing resources and storage space than cloud-based systems, confining the model size under a limitation. To better trade off the CM model sizes and performance, we proposed an adversarial speaker distillation method, which is an improved version of knowledge distillation method combined with generalized end-to-end (GE2E) pre-training and adversarial fine-tuning. In the evaluation phase of the ASVspoof 2021 Logical Access task, our proposed adversarial speaker distillation ResNetSE (ASD-ResNetSE) model reaches 0.2695 min t-DCF and 3.54% EER. ASD-ResNetSE only used 22.5% of parameters and 19.4% of multiply and accumulate operands of ResNetSE model.
Paper Structure (14 sections, 4 equations, 3 figures, 3 tables, 1 algorithm)

This paper contains 14 sections, 4 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methods
  3. Model structure
  4. Teacher training
  5. Generalized End-to-End Pre-training
  6. Adversarial fine-tuning
  7. Student training
  8. Experiment Setup
  9. Results and Analysis
  10. Ablation Study
  11. Overall Performance
  12. Model Size and Operands
  13. Conclusion
  14. Acknowledgements

Figures (3)

  • Figure 1: The proposed the training strategies for lightweight CM model for ASV. "Adv Dataset" represents the adversarial dataset generated by the AEG process.
  • Figure 2: Model structure of ResNetSE. (a) Sketch of ResNetSE model. (b) Details of Res-Block. (c) Details of squeeze-excitation layer.
  • Figure 3: Overall performance comparison.