Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Detecting Facial Image Manipulations with Multi-Layer CNN Models

Alejandro Marco Montejano, Angela Sanchez Perez, Javier Barrachina, David Ortiz-Perez, Manuel Benavent-Lledo, Jose Garcia-Rodriguez

TL;DR

This work tackles the challenge of detecting facial image manipulations generated by modern tools through lightweight, multi-layer CNNs derived from MesoNet. It introduces MesoNet+ for binary detection and MesoMultiNet/MesoMultiNet+ for multiclass manipulation classification, leveraging preprocessing (facial landmark alignment) and transfer learning from binary to multiclass models. Empirical results show MesoNet+ achieves about $0.90$ binary accuracy on unseen data, while MesoMultiNet+ reaches around $0.68$ overall accuracy for three classes, and incorporating AI-generated Stable Diffusion images raises MMN+ performance to about $0.76$, indicating strong gains from richer training data. The findings demonstrate the practical viability of low-computation CNNs for digital media verification and provide actionable guidance on architectural choices and training strategies for robust manipulation detection.

Abstract

The rapid evolution of digital image manipulation techniques poses significant challenges for content verification, with models such as stable diffusion and mid-journey producing highly realistic, yet synthetic, images that can deceive human perception. This research develops and evaluates convolutional neural networks (CNNs) specifically tailored for the detection of these manipulated images. The study implements a comparative analysis of three progressively complex CNN architectures, assessing their ability to classify and localize manipulations across various facial image modifications. Regularization and optimization techniques were systematically incorporated to improve feature extraction and performance. The results indicate that the proposed models achieve an accuracy of up to 76\% in distinguishing manipulated images from genuine ones, surpassing traditional approaches. This research not only highlights the potential of CNNs in enhancing the robustness of digital media verification tools, but also provides insights into effective architectural adaptations and training strategies for low-computation environments. Future work will build on these findings by extending the architectures to handle more diverse manipulation techniques and integrating multi-modal data for improved detection capabilities.

Detecting Facial Image Manipulations with Multi-Layer CNN Models

TL;DR

This work tackles the challenge of detecting facial image manipulations generated by modern tools through lightweight, multi-layer CNNs derived from MesoNet. It introduces MesoNet+ for binary detection and MesoMultiNet/MesoMultiNet+ for multiclass manipulation classification, leveraging preprocessing (facial landmark alignment) and transfer learning from binary to multiclass models. Empirical results show MesoNet+ achieves about binary accuracy on unseen data, while MesoMultiNet+ reaches around overall accuracy for three classes, and incorporating AI-generated Stable Diffusion images raises MMN+ performance to about , indicating strong gains from richer training data. The findings demonstrate the practical viability of low-computation CNNs for digital media verification and provide actionable guidance on architectural choices and training strategies for robust manipulation detection.

Abstract

The rapid evolution of digital image manipulation techniques poses significant challenges for content verification, with models such as stable diffusion and mid-journey producing highly realistic, yet synthetic, images that can deceive human perception. This research develops and evaluates convolutional neural networks (CNNs) specifically tailored for the detection of these manipulated images. The study implements a comparative analysis of three progressively complex CNN architectures, assessing their ability to classify and localize manipulations across various facial image modifications. Regularization and optimization techniques were systematically incorporated to improve feature extraction and performance. The results indicate that the proposed models achieve an accuracy of up to 76\% in distinguishing manipulated images from genuine ones, surpassing traditional approaches. This research not only highlights the potential of CNNs in enhancing the robustness of digital media verification tools, but also provides insights into effective architectural adaptations and training strategies for low-computation environments. Future work will build on these findings by extending the architectures to handle more diverse manipulation techniques and integrating multi-modal data for improved detection capabilities.

Paper Structure

This paper contains 19 sections, 9 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Current status of synthetic image generation and detection
  4. Potential of Multimodal Deep Learning Models for Detection and Localization of Manipulated Facial Images
  5. MesoNet. A compact network for facial video forgery detection
  6. Impact of Convolutional Filter Size and Number on Feature Extraction in the RGB Color Space Using MesoNet
  7. Methodology
  8. Binary Classification System: MesoNet+
  9. Multi-label classification system. MesoMultiNet
  10. Multiple classification system with 6 convolutional layers. MesoMultiNet+
  11. Evaluation of image classification models. Results and comparative analysis
  12. Dataset
  13. Preliminary Outcomes and Model Adaptations of MesoNet
  14. Evaluation and analysis of the results of the MesoNet+ binary classification model.
  15. Evaluation and performance analysis of the MesoMultiNet multi-class classification model
  16. ...and 4 more sections

Figures (9)

  • Figure 1: Comparison of original and aligned images for DeepFake and Bonafide classes.
  • Figure 2: 6-layer convolutional structure of MesoNet+.
  • Figure 3: True positive rate and false negative rate in the evaluation of the balanced MesoNET model.
  • Figure 4: True positive rate and false negative rate in the evaluation of the MesoNET model using an unknown dataset.
  • Figure 5: True positive rate and false negative rate in the evaluation of the MesoNET model using a balanced dataset.
  • ...and 4 more figures