Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HierCon: Hierarchical Contrastive Attention for Audio Deepfake Detection

Zhili Nicholas Liang, Soyeon Caren Han, Qizhou Wang, Christopher Leckie

TL;DR

This paper addresses the rising challenge of differentiating real from synthetic speech produced by modern TTS and VC systems. It introduces HierCon, a hierarchical attention framework that models temporal, intra-layer, and inter-layer dependencies in multi-layer SSL representations (e.g., XLS-R) and couples it with margin-based contrastive learning to produce domain-invariant embeddings. The approach achieves state-of-the-art results on ASVspoof 2021 DF and In-the-Wild benchmarks, with 1.93% EER and strong cross-domain generalisation, outperforming prior layer-weighting methods by substantial margins. Ablation studies confirm the complementary benefits of hierarchical attention and contrastive learning for robust generalisation, while interpretability analyses reveal meaningful multi-scale evidence driving decisions. The work advances robust audio deepfake detection with better generalisation across diverse generation pipelines and recording conditions, with potential for deployment in forensic and security contexts.

Abstract

Audio deepfakes generated by modern TTS and voice conversion systems are increasingly difficult to distinguish from real speech, raising serious risks for security and online trust. While state-of-the-art self-supervised models provide rich multi-layer representations, existing detectors treat layers independently and overlook temporal and hierarchical dependencies critical for identifying synthetic artefacts. We propose HierCon, a hierarchical layer attention framework combined with margin-based contrastive learning that models dependencies across temporal frames, neighbouring layers, and layer groups, while encouraging domain-invariant embeddings. Evaluated on ASVspoof 2021 DF and In-the-Wild datasets, our method achieves state-of-the-art performance (1.93% and 6.87% EER), improving over independent layer weighting by 36.6% and 22.5% respectively. The results and attention visualisations confirm that hierarchical modelling enhances generalisation to cross-domain generation techniques and recording conditions.

HierCon: Hierarchical Contrastive Attention for Audio Deepfake Detection

TL;DR

This paper addresses the rising challenge of differentiating real from synthetic speech produced by modern TTS and VC systems. It introduces HierCon, a hierarchical attention framework that models temporal, intra-layer, and inter-layer dependencies in multi-layer SSL representations (e.g., XLS-R) and couples it with margin-based contrastive learning to produce domain-invariant embeddings. The approach achieves state-of-the-art results on ASVspoof 2021 DF and In-the-Wild benchmarks, with 1.93% EER and strong cross-domain generalisation, outperforming prior layer-weighting methods by substantial margins. Ablation studies confirm the complementary benefits of hierarchical attention and contrastive learning for robust generalisation, while interpretability analyses reveal meaningful multi-scale evidence driving decisions. The work advances robust audio deepfake detection with better generalisation across diverse generation pipelines and recording conditions, with potential for deployment in forensic and security contexts.

Abstract

Audio deepfakes generated by modern TTS and voice conversion systems are increasingly difficult to distinguish from real speech, raising serious risks for security and online trust. While state-of-the-art self-supervised models provide rich multi-layer representations, existing detectors treat layers independently and overlook temporal and hierarchical dependencies critical for identifying synthetic artefacts. We propose HierCon, a hierarchical layer attention framework combined with margin-based contrastive learning that models dependencies across temporal frames, neighbouring layers, and layer groups, while encouraging domain-invariant embeddings. Evaluated on ASVspoof 2021 DF and In-the-Wild datasets, our method achieves state-of-the-art performance (1.93% and 6.87% EER), improving over independent layer weighting by 36.6% and 22.5% respectively. The results and attention visualisations confirm that hierarchical modelling enhances generalisation to cross-domain generation techniques and recording conditions.
Paper Structure (10 sections, 5 equations, 2 figures, 2 tables)

This paper contains 10 sections, 5 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Preliminaries
  4. HierCon
  5. Experiments
  6. Overall Performance
  7. Ablation Studies
  8. Attention Analysis and Interpretability
  9. Conclusion
  10. Acknowledgments

Figures (2)

  • Figure 1: Overall Architecture of the Proposed HierCon, a Hierarchical Contrastive Attention for the Reliable Audio Deepfake Detection.
  • Figure 2: HierCon attention averaged over 200 DF samples: (a) temporal focuses on mid frames (40--70%); (b) intra-group shifts from shallow (L0: 0.438) to deep (L2: 0.572); (c) inter-group peaks at Group 5 (L12--14: 0.243).