Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evolutionary Multi-Objective Fusion of Deepfake Speech Detectors

Vojtěch Staněk, Martin Perešíni, Lukáš Sekanina, Anton Firc, Kamil Malinka

Abstract

While deepfake speech detectors built on large self-supervised learning (SSL) models achieve high accuracy, employing standard ensemble fusion to further enhance robustness often results in oversized systems with diminishing returns. To address this, we propose an evolutionary multi-objective score fusion framework that jointly minimizes detection error and system complexity. We explore two encodings optimized by NSGA-II: binary-coded detector selection for score averaging and a real-valued scheme that optimizes detector weights for a weighted sum. Experiments on the ASVspoof 5 dataset with 36 SSL-based detectors show that the obtained Pareto fronts outperform simple averaging and logistic regression baselines. The real-valued variant achieves 2.37% EER (0.0684 minDCF) and identifies configurations that match state-of-the-art performance while significantly reducing system complexity, requiring only half the parameters. Our method also provides a diverse set of trade-off solutions, enabling deployment choices that balance accuracy and computational cost.

Evolutionary Multi-Objective Fusion of Deepfake Speech Detectors

Abstract

While deepfake speech detectors built on large self-supervised learning (SSL) models achieve high accuracy, employing standard ensemble fusion to further enhance robustness often results in oversized systems with diminishing returns. To address this, we propose an evolutionary multi-objective score fusion framework that jointly minimizes detection error and system complexity. We explore two encodings optimized by NSGA-II: binary-coded detector selection for score averaging and a real-valued scheme that optimizes detector weights for a weighted sum. Experiments on the ASVspoof 5 dataset with 36 SSL-based detectors show that the obtained Pareto fronts outperform simple averaging and logistic regression baselines. The real-valued variant achieves 2.37% EER (0.0684 minDCF) and identifies configurations that match state-of-the-art performance while significantly reducing system complexity, requiring only half the parameters. Our method also provides a diverse set of trade-off solutions, enabling deployment choices that balance accuracy and computational cost.

Paper Structure

This paper contains 16 sections, 3 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Deepfake Speech Detection
  4. Ensemble Learning and Classifier Fusion
  5. Evolutionary Machine Learning
  6. Proposed Multi-Objective Fusion Approach
  7. Fusion objectives
  8. Evolutionary optimization mechanism
  9. Experiment setup
  10. Base detectors
  11. Baseline fusions
  12. Fusions Optimized with NSGA-II
  13. Results
  14. Comparison with State-of-the-Art
  15. Discussion
  16. ...and 1 more sections

Figures (7)

  • Figure 1: Overview of the proposed evolutionary multi-objective fusion framework. The system processes input recordings through a pool of base detectors to obtain scores and parameter counts. A candidate fusion is constructed and evaluated based on a randomly initialized chromosome. NSGA-II then iteratively optimizes the fusion configuration (i.e., the chromosome) to simultaneously minimize the Equal Error Rate (EER) and system complexity (i.e., the number of parameters), ultimately producing a Pareto front of optimal trade-off solutions.
  • Figure 2: Chromosome representations for the two variants.
  • Figure 3: Impact of population size on HV convergence under fixed computational budget (best viewed in color). Averages of 10 runs are reported.
  • Figure 4: HV convergence -- averages of 10 runs are reported.
  • Figure 5: Parameter sensitivity analysis for the real-valued variant: mean HV vs. $p_m$ and $p_c$ for three distinct distribution indices ($\eta_m$) under a fixed computational budget of 25,000 fitness evaluations.
  • ...and 2 more figures