Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Audio Deepfake Detection at the First Greeting: "Hi!"

Haohan Shi, Xiyu Shi, Safak Dogan, Tianjin Huang, Yunxiao Zhang

TL;DR

This work addresses the challenge of detecting audio deepfakes from ultra-short utterances (0.5–2.0 seconds) under real-world communication degradations. It introduces Short-MGAA (S-MGAA), a lightweight framework that extends MGAA with two specialized modules: Pixel-Channel Enhanced Module (PCEM) and Frequency Compensation Enhanced Module (FCEM), to improve discriminative representation learning when temporal evidence is scarce. PCEM amplifies fine-grained time-frequency saliency while FCEM exploits multi-scale frequency analysis and adaptive frequency-temporal interaction to compensate limited temporal cues. Empirical results on a large, degraded dataset show that S-MGAA consistently surpasses nine state-of-the-art baselines across durations, with substantial EER reductions, and maintains a favorable efficiency-accuracy balance suitable for real-time, edge-device deployment.

Abstract

This paper focuses on audio deepfake detection under real-world communication degradations, with an emphasis on ultra-short inputs (0.5-2.0s), targeting the capability to detect synthetic speech at a conversation opening, e.g., when a scammer says "Hi." We propose Short-MGAA (S-MGAA), a novel lightweight extension of Multi-Granularity Adaptive Time-Frequency Attention, designed to enhance discriminative representation learning for short, degraded inputs subjected to communication processing and perturbations. The S-MGAA integrates two tailored modules: a Pixel-Channel Enhanced Module (PCEM) that amplifies fine-grained time-frequency saliency, and a Frequency Compensation Enhanced Module (FCEM) to supplement limited temporal evidence via multi-scale frequency modeling and adaptive frequency-temporal interaction. Extensive experiments demonstrate that S-MGAA consistently surpasses nine state-of-the-art baselines while achieving strong robustness to degradations and favorable efficiency-accuracy trade-offs, including low RTF, competitive GFLOPs, compact parameters, and reduced training cost, highlighting its strong potential for real-time deployment in communication systems and edge devices.

Audio Deepfake Detection at the First Greeting: "Hi!"

TL;DR

This work addresses the challenge of detecting audio deepfakes from ultra-short utterances (0.5–2.0 seconds) under real-world communication degradations. It introduces Short-MGAA (S-MGAA), a lightweight framework that extends MGAA with two specialized modules: Pixel-Channel Enhanced Module (PCEM) and Frequency Compensation Enhanced Module (FCEM), to improve discriminative representation learning when temporal evidence is scarce. PCEM amplifies fine-grained time-frequency saliency while FCEM exploits multi-scale frequency analysis and adaptive frequency-temporal interaction to compensate limited temporal cues. Empirical results on a large, degraded dataset show that S-MGAA consistently surpasses nine state-of-the-art baselines across durations, with substantial EER reductions, and maintains a favorable efficiency-accuracy balance suitable for real-time, edge-device deployment.

Abstract

This paper focuses on audio deepfake detection under real-world communication degradations, with an emphasis on ultra-short inputs (0.5-2.0s), targeting the capability to detect synthetic speech at a conversation opening, e.g., when a scammer says "Hi." We propose Short-MGAA (S-MGAA), a novel lightweight extension of Multi-Granularity Adaptive Time-Frequency Attention, designed to enhance discriminative representation learning for short, degraded inputs subjected to communication processing and perturbations. The S-MGAA integrates two tailored modules: a Pixel-Channel Enhanced Module (PCEM) that amplifies fine-grained time-frequency saliency, and a Frequency Compensation Enhanced Module (FCEM) to supplement limited temporal evidence via multi-scale frequency modeling and adaptive frequency-temporal interaction. Extensive experiments demonstrate that S-MGAA consistently surpasses nine state-of-the-art baselines while achieving strong robustness to degradations and favorable efficiency-accuracy trade-offs, including low RTF, competitive GFLOPs, compact parameters, and reduced training cost, highlighting its strong potential for real-time deployment in communication systems and edge devices.
Paper Structure (12 sections, 6 equations, 3 figures, 3 tables)

This paper contains 12 sections, 6 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Framework Architecture
  4. Pixel-Channel Enhanced Module (PCEM)
  5. Frequency Compensation Enhanced Module (FCEM)
  6. Experiments
  7. Dataset and Metrics
  8. Implementation Details and Baselines
  9. Experimental Results
  10. Ablation Study
  11. Complexity and Deployability Analysis
  12. Conclusion

Figures (3)

  • Figure 1: Proposed ADD framework for ultra-short-duration audio inputs. (a) The processing pipeline; (b) Short Multi-Granularity Adaptive Time-Frequency Attention; (c) Convolutional Feature Embedding Blocks; (d) The Classifier.
  • Figure 2: Average EER (%) of baselines across the $C_0$–$C_5$ conditions for audio durations of 0.5s, 1s, 1.5s, 2s and 4s.
  • Figure 3: Efficiency comparison across 0.5–2.0s. Radar plots show log‑scaled, per‑duration–normalized metrics (outer position is better after inversion for RTF, training time, GFLOPs, and parameters). Green coloured areas denote the proposed S‑MGAA variants.