Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Herd Mentality in Augmentation -- Not a Good Idea! A Robust Multi-stage Approach towards Deepfake Detection

Monu, Rohan Raju Dhanakshirur

TL;DR

An enhanced architecture based on the GenConViT model, which incorporates weighted loss and update augmentation techniques and includes masked eye pretraining is proposed, which improves the F1 score and the accuracy on the Celeb-DF v2 dataset.

Abstract

The rapid increase in deepfake technology has raised significant concerns about digital media integrity. Detecting deepfakes is crucial for safeguarding digital media. However, most standard image classifiers fail to distinguish between fake and real faces. Our analysis reveals that this failure is due to the model's inability to explicitly focus on the artefacts typically in deepfakes. We propose an enhanced architecture based on the GenConViT model, which incorporates weighted loss and update augmentation techniques and includes masked eye pretraining. This proposed model improves the F1 score by 1.71% and the accuracy by 4.34% on the Celeb-DF v2 dataset. The source code for our model is available at https://github.com/Monu-Khicher-1/multi-stage-learning

Herd Mentality in Augmentation -- Not a Good Idea! A Robust Multi-stage Approach towards Deepfake Detection

TL;DR

An enhanced architecture based on the GenConViT model, which incorporates weighted loss and update augmentation techniques and includes masked eye pretraining is proposed, which improves the F1 score and the accuracy on the Celeb-DF v2 dataset.

Abstract

The rapid increase in deepfake technology has raised significant concerns about digital media integrity. Detecting deepfakes is crucial for safeguarding digital media. However, most standard image classifiers fail to distinguish between fake and real faces. Our analysis reveals that this failure is due to the model's inability to explicitly focus on the artefacts typically in deepfakes. We propose an enhanced architecture based on the GenConViT model, which incorporates weighted loss and update augmentation techniques and includes masked eye pretraining. This proposed model improves the F1 score by 1.71% and the accuracy by 4.34% on the Celeb-DF v2 dataset. The source code for our model is available at https://github.com/Monu-Khicher-1/multi-stage-learning
Paper Structure (13 sections, 1 equation, 2 figures, 2 tables)

This paper contains 13 sections, 1 equation, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Proposed Architecture
  3. Backbone Architecture:
  4. Problems with GenConViT:
  5. Modified Data Augmentation(NA & RF):
  6. Hardness-inspired Curriculum Learning (MEP):
  7. Weighted Loss Function (WL):
  8. Results and Discussion
  9. Dataset under consideration:
  10. Evaluation Metrics:
  11. Quantitative Results on CelebDF:
  12. Ablation Analysis:
  13. Conclusion

Figures (2)

  • Figure 1: (a) State of the art data preprocessing as used in cite9 (b) Baseline cite9 model architecture
  • Figure 2: Architecture analyses with GradCAM