Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deepfake Detection of Face Images based on a Convolutional Neural Network

Lukas Kroiß, Johannes Reschke

TL;DR

This work tackles deepfake detection in face images by fine-tuning a ResNet-50 backbone on the Diverse Fake Face Dataset (DFFD), replacing the final classifier with a flatten layer and a single sigmoid output for binary real/fake decisions. The approach employs transfer learning in a three-step process and evaluates predictions through ROC analysis, achieving an extremely high AUC of $0.9931$ and a F1-score of $0.9715$ on a held-out test set. The results demonstrate robust discrimination with a well-chosen threshold ($th=0.6587$) and provide insights via confusion matrices and probability-density plots. The findings suggest that full-network fine-tuning offers limited gains beyond the initial training, highlighting practical implications for deploying efficient deepfake detectors with strong performance in real-world settings.

Abstract

Fake News and especially deepfakes (generated, non-real image or video content) have become a serious topic over the last years. With the emergence of machine learning algorithms it is now easier than ever before to generate such fake content, even for private persons. This issue of generated fake images is especially critical in the context of politics and public figures. We want to address this conflict by building a model based on a Convolutions Neural Network in order to detect such generated and fake images showing human portraits. As a basis, we use a pre-trained ResNet-50 model due to its effectiveness in terms of classifying images. We then adopted the base model to our task of classifying a single image as authentic/real or fake by adding an fully connected output layer containing a single neuron indicating the authenticity of an image. We applied fine tuning and transfer learning to develop the model and improve its parameters. For the training process we collected the image data set "Diverse Face Fake Dataset" containing a wide range of different image manipulation methods and also diversity in terms of faces visible on the images. With our final model we reached the following outstanding performance metrics: precision = 0.98, recall 0.96, F1-Score = 0.97 and an area-under-curve = 0.99.

Deepfake Detection of Face Images based on a Convolutional Neural Network

TL;DR

This work tackles deepfake detection in face images by fine-tuning a ResNet-50 backbone on the Diverse Fake Face Dataset (DFFD), replacing the final classifier with a flatten layer and a single sigmoid output for binary real/fake decisions. The approach employs transfer learning in a three-step process and evaluates predictions through ROC analysis, achieving an extremely high AUC of and a F1-score of on a held-out test set. The results demonstrate robust discrimination with a well-chosen threshold () and provide insights via confusion matrices and probability-density plots. The findings suggest that full-network fine-tuning offers limited gains beyond the initial training, highlighting practical implications for deploying efficient deepfake detectors with strong performance in real-world settings.

Abstract

Fake News and especially deepfakes (generated, non-real image or video content) have become a serious topic over the last years. With the emergence of machine learning algorithms it is now easier than ever before to generate such fake content, even for private persons. This issue of generated fake images is especially critical in the context of politics and public figures. We want to address this conflict by building a model based on a Convolutions Neural Network in order to detect such generated and fake images showing human portraits. As a basis, we use a pre-trained ResNet-50 model due to its effectiveness in terms of classifying images. We then adopted the base model to our task of classifying a single image as authentic/real or fake by adding an fully connected output layer containing a single neuron indicating the authenticity of an image. We applied fine tuning and transfer learning to develop the model and improve its parameters. For the training process we collected the image data set "Diverse Face Fake Dataset" containing a wide range of different image manipulation methods and also diversity in terms of faces visible on the images. With our final model we reached the following outstanding performance metrics: precision = 0.98, recall 0.96, F1-Score = 0.97 and an area-under-curve = 0.99.

Paper Structure

This paper contains 27 sections, 12 equations, 13 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Model
  4. Dataset
  5. Transfer Learning
  6. Step 1
  7. Step 2
  8. Step 3
  9. Interpretation of Model Prediction
  10. Results
  11. Receiver Operating Characteristic (ROC)
  12. Confusion Matrix
  13. F1-Score
  14. Probability Density Function
  15. Discussion
  16. ...and 12 more sections

Figures (13)

  • Figure 1: Residual bottleneck building block with shortcut connection key:He_2015
  • Figure 2: Model architecture of own adopted neural network
  • Figure 3: Training Process Step 1
  • Figure 4: Training Process Step 2
  • Figure 5: Training Process Step 3
  • ...and 8 more figures