Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unmasking Facial DeepFakes: A Robust Multiview Detection Framework for Natural Images

Sami Belguesmia, Mohand Saïd Allili, Assia Hamadene

TL;DR

This work tackles robust DeepFake detection under pose variation and occlusion by proposing a multi-view architecture that jointly analyzes global, middle, and local facial features plus an explicit face-orientation encoder. Features from the four branches are fused via an MLP to produce a detection decision, with CNN and Transformer variants explored. Experiments on OpenForensics and FaceForensics++ show that multi-view fusion plus orientation information outperforms single-view baselines and competes with or surpasses recent state-of-the-art methods, demonstrating improved robustness in unconstrained settings. The work highlights the value of region-specific artifacts and pose-aware representations for real-world forensics, and opens paths toward temporal modeling and explainability.

Abstract

DeepFake technology has advanced significantly in recent years, enabling the creation of highly realistic synthetic face images. Existing DeepFake detection methods often struggle with pose variations, occlusions, and artifacts that are difficult to detect in real-world conditions. To address these challenges, we propose a multi-view architecture that enhances DeepFake detection by analyzing facial features at multiple levels. Our approach integrates three specialized encoders, a global view encoder for detecting boundary inconsistencies, a middle view encoder for analyzing texture and color alignment, and a local view encoder for capturing distortions in expressive facial regions such as the eyes, nose, and mouth, where DeepFake artifacts frequently occur. Additionally, we incorporate a face orientation encoder, trained to classify face poses, ensuring robust detection across various viewing angles. By fusing features from these encoders, our model achieves superior performance in detecting manipulated images, even under challenging pose and lighting conditions.Experimental results on challenging datasets demonstrate the effectiveness of our method, outperforming conventional single-view approaches

Unmasking Facial DeepFakes: A Robust Multiview Detection Framework for Natural Images

TL;DR

This work tackles robust DeepFake detection under pose variation and occlusion by proposing a multi-view architecture that jointly analyzes global, middle, and local facial features plus an explicit face-orientation encoder. Features from the four branches are fused via an MLP to produce a detection decision, with CNN and Transformer variants explored. Experiments on OpenForensics and FaceForensics++ show that multi-view fusion plus orientation information outperforms single-view baselines and competes with or surpasses recent state-of-the-art methods, demonstrating improved robustness in unconstrained settings. The work highlights the value of region-specific artifacts and pose-aware representations for real-world forensics, and opens paths toward temporal modeling and explainability.

Abstract

DeepFake technology has advanced significantly in recent years, enabling the creation of highly realistic synthetic face images. Existing DeepFake detection methods often struggle with pose variations, occlusions, and artifacts that are difficult to detect in real-world conditions. To address these challenges, we propose a multi-view architecture that enhances DeepFake detection by analyzing facial features at multiple levels. Our approach integrates three specialized encoders, a global view encoder for detecting boundary inconsistencies, a middle view encoder for analyzing texture and color alignment, and a local view encoder for capturing distortions in expressive facial regions such as the eyes, nose, and mouth, where DeepFake artifacts frequently occur. Additionally, we incorporate a face orientation encoder, trained to classify face poses, ensuring robust detection across various viewing angles. By fusing features from these encoders, our model achieves superior performance in detecting manipulated images, even under challenging pose and lighting conditions.Experimental results on challenging datasets demonstrate the effectiveness of our method, outperforming conventional single-view approaches
Paper Structure (12 sections, 5 figures, 2 tables)

This paper contains 12 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Methodology
  4. Image pre-processing
  5. The general multi-view architecture
  6. Model initialization and training
  7. Experimental results
  8. Datasets
  9. Quantitative results
  10. Qualitative results
  11. Conclusion
  12. Acknowledgment

Figures (5)

  • Figure 1: Illustration of the image pre-processing creating : (b) global, (c) middle, and (d) local views, respectively, from the original image (a).
  • Figure 2: Overview of the proposed three stream architecture. The global and middle stream extract holistic facial representations, while the local region-specific stream focuses on high-importance facial sub-regions.
  • Figure 3: Illustration of fusion module integrating local view (LV), middle view (MV), global view (GV) and pose features.
  • Figure 4: Samples from different DeepFake datasets. From left to right, FaceForensics++ rossler2019faceforensics, DFDC dolhansky2020deepfake, DeeperForensics jiang2020deeperforensics, Celeb-DF li2020celebdf and OpenForensics Le2021.
  • Figure 5: Examples of GradCam calculation for two images using the BeiT (middle column) and CNN (right column) implementations.