Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fair and Interpretable Deepfake Detection in Videos

Akihito Yoshii, Ryosuke Sonoda, Ramya Srinivasan

TL;DR

We address fairness in deepfake video detection by integrating temporal feature learning with a concept-based bias analysis framework. A Concept Bank and temporal clustering identify demographic biases, quantified via Concept Sensitivity Score, while a bias-aware sampling strategy guides training alongside a frequency-aware data augmentation method that preserves high-frequency deepfake artifacts. Empirical results on FF++ , DFD, Celeb-DF, and DFDC using Xception and ResNet backbones show improved fairness without sacrificing accuracy, with ablations confirming the value of both the concept-extraction and augmentation components. This approach enhances interpretability through concept-based explanations and improves generalization across demographic groups, offering practical benefits for real-world deployment of deepfake detectors.

Abstract

Existing deepfake detection methods often exhibit bias, lack transparency, and fail to capture temporal information, leading to biased decisions and unreliable results across different demographic groups. In this paper, we propose a fairness-aware deepfake detection framework that integrates temporal feature learning and demographic-aware data augmentation to enhance fairness and interpretability. Our method leverages sequence-based clustering for temporal modeling of deepfake videos and concept extraction to improve detection reliability while also facilitating interpretable decisions for non-expert users. Additionally, we introduce a demography-aware data augmentation method that balances underrepresented groups and applies frequency-domain transformations to preserve deepfake artifacts, thereby mitigating bias and improving generalization. Extensive experiments on FaceForensics++, DFD, Celeb-DF, and DFDC datasets using state-of-the-art (SoTA) architectures (Xception, ResNet) demonstrate the efficacy of the proposed method in obtaining the best tradeoff between fairness and accuracy when compared to SoTA.

Fair and Interpretable Deepfake Detection in Videos

TL;DR

We address fairness in deepfake video detection by integrating temporal feature learning with a concept-based bias analysis framework. A Concept Bank and temporal clustering identify demographic biases, quantified via Concept Sensitivity Score, while a bias-aware sampling strategy guides training alongside a frequency-aware data augmentation method that preserves high-frequency deepfake artifacts. Empirical results on FF++ , DFD, Celeb-DF, and DFDC using Xception and ResNet backbones show improved fairness without sacrificing accuracy, with ablations confirming the value of both the concept-extraction and augmentation components. This approach enhances interpretability through concept-based explanations and improves generalization across demographic groups, offering practical benefits for real-world deployment of deepfake detectors.

Abstract

Existing deepfake detection methods often exhibit bias, lack transparency, and fail to capture temporal information, leading to biased decisions and unreliable results across different demographic groups. In this paper, we propose a fairness-aware deepfake detection framework that integrates temporal feature learning and demographic-aware data augmentation to enhance fairness and interpretability. Our method leverages sequence-based clustering for temporal modeling of deepfake videos and concept extraction to improve detection reliability while also facilitating interpretable decisions for non-expert users. Additionally, we introduce a demography-aware data augmentation method that balances underrepresented groups and applies frequency-domain transformations to preserve deepfake artifacts, thereby mitigating bias and improving generalization. Extensive experiments on FaceForensics++, DFD, Celeb-DF, and DFDC datasets using state-of-the-art (SoTA) architectures (Xception, ResNet) demonstrate the efficacy of the proposed method in obtaining the best tradeoff between fairness and accuracy when compared to SoTA.
Paper Structure (20 sections, 16 equations, 2 figures, 4 tables, 1 algorithm)

This paper contains 20 sections, 16 equations, 2 figures, 4 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. Related work
  3. Deepfake Detection in Videos
  4. Bias Mitigation in Deepfake Detection
  5. Transparency in Deepfake Detection
  6. Method
  7. Proposed Concept Extraction Method
  8. Concept Bank Construction
  9. Identification of Concept-based Bias
  10. Clustering with Temporal Information.
  11. Quantifying Bias Using Concept Sensitivity Score (CSS).
  12. Bias-aware Sampling Strategy
  13. Connection with Data Augmentation
  14. Proposed Data Augmentation Method
  15. Experiment
  16. ...and 5 more sections

Figures (2)

  • Figure 1: Illustration of system outputs for an example input video. Five frames with heat maps highlighting the likelihood of fake regions and Concept Sensitivity Score (CSS) providing human-interpretable explanations of model's decision.
  • Figure 2: System diagram. Our framework first extracts proxy attributes for demographic attributes from the training data. Next, it applies frequency-aware data mixing to mitigate biases associated with these attributes. Finally, the model is re-trained on the de-biased dataset.