Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Data-Driven Diffusion-based Approach for Audio Deepfake Explanations

Petr Grinberg, Ankur Kumar, Surya Koppisetti, Gaurav Bharaj

TL;DR

This paper tackles the challenge of explainability for audio deepfake detection (ADD) by exploiting ground-truth artifact regions available from paired real and vocoded audio. It introduces a data-driven XAI framework that uses the spectrogram difference as supervision to train a diffusion model to generate time-frequency attribution heatmaps. Two conditioning variants are proposed: SpecSegDiff, which conditions on spoof spectrograms, and ADDSegDiff, which conditions on intermediate ADD features, both evaluated against classical XAI baselines on VocV4 and LibriSeVoc. Results show that diffusion-based explanations align more faithfully with ground truth and the decision process, offering a robust path toward faithful, task-specific audio explanations.

Abstract

Evaluating explainability techniques, such as SHAP and LRP, in the context of audio deepfake detection is challenging due to lack of clear ground truth annotations. In the cases when we are able to obtain the ground truth, we find that these methods struggle to provide accurate explanations. In this work, we propose a novel data-driven approach to identify artifact regions in deepfake audio. We consider paired real and vocoded audio, and use the difference in time-frequency representation as the ground-truth explanation. The difference signal then serves as a supervision to train a diffusion model to expose the deepfake artifacts in a given vocoded audio. Experimental results on the VocV4 and LibriSeVoc datasets demonstrate that our method outperforms traditional explainability techniques, both qualitatively and quantitatively.

A Data-Driven Diffusion-based Approach for Audio Deepfake Explanations

TL;DR

This paper tackles the challenge of explainability for audio deepfake detection (ADD) by exploiting ground-truth artifact regions available from paired real and vocoded audio. It introduces a data-driven XAI framework that uses the spectrogram difference as supervision to train a diffusion model to generate time-frequency attribution heatmaps. Two conditioning variants are proposed: SpecSegDiff, which conditions on spoof spectrograms, and ADDSegDiff, which conditions on intermediate ADD features, both evaluated against classical XAI baselines on VocV4 and LibriSeVoc. Results show that diffusion-based explanations align more faithfully with ground truth and the decision process, offering a robust path toward faithful, task-specific audio explanations.

Abstract

Evaluating explainability techniques, such as SHAP and LRP, in the context of audio deepfake detection is challenging due to lack of clear ground truth annotations. In the cases when we are able to obtain the ground truth, we find that these methods struggle to provide accurate explanations. In this work, we propose a novel data-driven approach to identify artifact regions in deepfake audio. We consider paired real and vocoded audio, and use the difference in time-frequency representation as the ground-truth explanation. The difference signal then serves as a supervision to train a diffusion model to expose the deepfake artifacts in a given vocoded audio. Experimental results on the VocV4 and LibriSeVoc datasets demonstrate that our method outperforms traditional explainability techniques, both qualitatively and quantitatively.

Paper Structure

This paper contains 11 sections, 2 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Classical explainability methods
  5. Dataset for supervised XAI
  6. Diffusion-based XAI
  7. Evaluation criteria
  8. Experiments
  9. Experimental setup
  10. Results and discussion
  11. Conclusion

Figures (3)

  • Figure 1: Original (top) and smoothed (bottom) versions of bona fide and spoof spectrograms for a parallel pair in VocV4 wang2023spoofed, with corresponding segmentation masks (right). Rectangles show manipulated regions: (1) missed harmonics, (2) energy loss.
  • Figure 2: Proposed diffusion model framework to predict artifact regions in spoof audio. For conditioning, we use the spectrogram in SpecSegDiff, whereas, in ADDSegDiff, the intermediate features from a pretrained frozen ADD model are used.
  • Figure 3: Comparison of the attributions from classic XAI tools and our diffusion model. We show binarized (B) and heatmap (H) versions. The binarization uses $95\%$ quantile threshold.