Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explainable Deepfake Video Detection using Convolutional Neural Network and CapsuleNet

Gazi Hasin Ishrak, Zalish Mahmud, MD. Zami Al Zunaed Farabe, Tahera Khanom Tinni, Tanzim Reza, Mohammad Zavid Parvez

TL;DR

This work tackles deepfake detection by pairing a CapsuleNet-based architecture with ConvLSTM and pre-trained CNN backbones to capture both spatial and temporal cues, augmented with Grad-CAM explanations for model transparency. Using the DFDC dataset, it preprocesses and labels frames, then trains a hybrid CapsuleNet-LSTM model that includes a ConvLSTM_2D layer, a Conv2D layer, primary and secondary capsule layers, and a Sequential LSTM, followed by dense layers. The study reports a validation accuracy of 88% and an AUC of 0.951 on full-frame data, with explainability via Grad-CAM showing attention to facial regions in real videos and attenuated signals in fake ones, outperforming a combined TemperNet/XceptionNet baseline. The approach demonstrates practical potential for robust, explainable deepfake detection in real-world scenarios, balancing accuracy with interpretability.

Abstract

Deepfake technology, derived from deep learning, seamlessly inserts individuals into digital media, irrespective of their actual participation. Its foundation lies in machine learning and Artificial Intelligence (AI). Initially, deepfakes served research, industry, and entertainment. While the concept has existed for decades, recent advancements render deepfakes nearly indistinguishable from reality. Accessibility has soared, empowering even novices to create convincing deepfakes. However, this accessibility raises security concerns.The primary deepfake creation algorithm, GAN (Generative Adversarial Network), employs machine learning to craft realistic images or videos. Our objective is to utilize CNN (Convolutional Neural Network) and CapsuleNet with LSTM to differentiate between deepfake-generated frames and originals. Furthermore, we aim to elucidate our model's decision-making process through Explainable AI, fostering transparent human-AI relationships and offering practical examples for real-life scenarios.

Explainable Deepfake Video Detection using Convolutional Neural Network and CapsuleNet

TL;DR

This work tackles deepfake detection by pairing a CapsuleNet-based architecture with ConvLSTM and pre-trained CNN backbones to capture both spatial and temporal cues, augmented with Grad-CAM explanations for model transparency. Using the DFDC dataset, it preprocesses and labels frames, then trains a hybrid CapsuleNet-LSTM model that includes a ConvLSTM_2D layer, a Conv2D layer, primary and secondary capsule layers, and a Sequential LSTM, followed by dense layers. The study reports a validation accuracy of 88% and an AUC of 0.951 on full-frame data, with explainability via Grad-CAM showing attention to facial regions in real videos and attenuated signals in fake ones, outperforming a combined TemperNet/XceptionNet baseline. The approach demonstrates practical potential for robust, explainable deepfake detection in real-world scenarios, balancing accuracy with interpretability.

Abstract

Deepfake technology, derived from deep learning, seamlessly inserts individuals into digital media, irrespective of their actual participation. Its foundation lies in machine learning and Artificial Intelligence (AI). Initially, deepfakes served research, industry, and entertainment. While the concept has existed for decades, recent advancements render deepfakes nearly indistinguishable from reality. Accessibility has soared, empowering even novices to create convincing deepfakes. However, this accessibility raises security concerns.The primary deepfake creation algorithm, GAN (Generative Adversarial Network), employs machine learning to craft realistic images or videos. Our objective is to utilize CNN (Convolutional Neural Network) and CapsuleNet with LSTM to differentiate between deepfake-generated frames and originals. Furthermore, we aim to elucidate our model's decision-making process through Explainable AI, fostering transparent human-AI relationships and offering practical examples for real-life scenarios.
Paper Structure (34 sections, 2 equations, 8 figures, 7 tables)

This paper contains 34 sections, 2 equations, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Deepfake
  4. Deepfake techniques
  5. Face swap
  6. Face re-enactment
  7. Adversarial methods
  8. AI-based Video Synthesis Algorithm
  9. Deepfake detection
  10. Mesonet
  11. XceptionNet
  12. Convolutional LSTM
  13. RecurrentConvolutional
  14. Resampling Detection
  15. FakeSpotter
  16. ...and 19 more sections

Figures (8)

  • Figure 1: Proposed Methodology Diagram
  • Figure 2: Data Pre-processing strategy
  • Figure 3: Proposed Hybrid Model
  • Figure 4: History of Accuracy
  • Figure 5: History of Loss
  • ...and 3 more figures