Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Classifying Deepfakes Using Swin Transformers

Aprille J. Xi, Eason Chen

TL;DR

This work tackles image-based deepfake detection by evaluating Swin Transformer architectures, including standalone Swin and hybrids like Swin-ResNet18, on the Real and Fake Face Detection dataset. It integrates Error Level Analysis (ELA) preprocessing and compares against CNN baselines (VGG16, ResNet18, AlexNet), highlighting Swin's ability to capture both global and local manipulation cues. The standalone Swin Transformer achieves the highest test accuracy of 71.29%, while some hybrids show gains or fail due to overfitting, underscoring the potential and challenges of transformer-based approaches. Overall, the study demonstrates that transformer architectures can improve accuracy and generalization in image-based deepfake detection, informing future research on robust countermeasures against manipulated media.

Abstract

The proliferation of deepfake technology poses significant challenges to the authenticity and trustworthiness of digital media, necessitating the development of robust detection methods. This study explores the application of Swin Transformers, a state-of-the-art architecture leveraging shifted windows for self-attention, in detecting and classifying deepfake images. Using the Real and Fake Face Detection dataset by Yonsei University's Computational Intelligence Photography Lab, we evaluate the Swin Transformer and hybrid models such as Swin-ResNet and Swin-KNN, focusing on their ability to identify subtle manipulation artifacts. Our results demonstrate that the Swin Transformer outperforms conventional CNN-based architectures, including VGG16, ResNet18, and AlexNet, achieving a test accuracy of 71.29%. Additionally, we present insights into hybrid model design, highlighting the complementary strengths of transformer and CNN-based approaches in deepfake detection. This study underscores the potential of transformer-based architectures for improving accuracy and generalizability in image-based manipulation detection, paving the way for more effective countermeasures against deepfake threats.

Classifying Deepfakes Using Swin Transformers

TL;DR

This work tackles image-based deepfake detection by evaluating Swin Transformer architectures, including standalone Swin and hybrids like Swin-ResNet18, on the Real and Fake Face Detection dataset. It integrates Error Level Analysis (ELA) preprocessing and compares against CNN baselines (VGG16, ResNet18, AlexNet), highlighting Swin's ability to capture both global and local manipulation cues. The standalone Swin Transformer achieves the highest test accuracy of 71.29%, while some hybrids show gains or fail due to overfitting, underscoring the potential and challenges of transformer-based approaches. Overall, the study demonstrates that transformer architectures can improve accuracy and generalization in image-based deepfake detection, informing future research on robust countermeasures against manipulated media.

Abstract

The proliferation of deepfake technology poses significant challenges to the authenticity and trustworthiness of digital media, necessitating the development of robust detection methods. This study explores the application of Swin Transformers, a state-of-the-art architecture leveraging shifted windows for self-attention, in detecting and classifying deepfake images. Using the Real and Fake Face Detection dataset by Yonsei University's Computational Intelligence Photography Lab, we evaluate the Swin Transformer and hybrid models such as Swin-ResNet and Swin-KNN, focusing on their ability to identify subtle manipulation artifacts. Our results demonstrate that the Swin Transformer outperforms conventional CNN-based architectures, including VGG16, ResNet18, and AlexNet, achieving a test accuracy of 71.29%. Additionally, we present insights into hybrid model design, highlighting the complementary strengths of transformer and CNN-based approaches in deepfake detection. This study underscores the potential of transformer-based architectures for improving accuracy and generalizability in image-based manipulation detection, paving the way for more effective countermeasures against deepfake threats.
Paper Structure (5 sections, 2 equations, 1 figure, 1 table)

This paper contains 5 sections, 2 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Results
  5. Discussion, Limitation, and Future Works

Figures (1)

  • Figure 1: Training and testing performance comparison of various deepfake classification models, including AlexNet, ResNet18, VGG16, Swin, and hybrid architectures. The top row shows training loss and accuracy, while the bottom row shows testing loss and accuracy.