Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring the Impact of Moire Pattern on Deepfake Detectors

Razaib Tariq, Shahroz Tariq, Simon S. Woo

TL;DR

This paper investigates how Moiré patterns, arising when camera-captured deepfakes are viewed on screens, degrade deepfake detectors deployed on external devices. Using camera-captured videos from CelebDF and FF++ and six state-of-the-art detectors, the study reveals substantial performance loss, with average F1-scores not exceeding 68% and some detectors performing as low as 47%. AUROC analyses show particularly weak discriminative power under Moiré artifacts, with CLRNet reaching only 0.64 on CelebDF. The work highlights the need for Moiré-aware preprocessing, diversified datasets, and cross-domain evaluation to enhance the real-world robustness of deepfake detection systems.

Abstract

Deepfake detection is critical in mitigating the societal threats posed by manipulated videos. While various algorithms have been developed for this purpose, challenges arise when detectors operate externally, such as on smartphones, when users take a photo of deepfake images and upload on the Internet. One significant challenge in such scenarios is the presence of Moiré patterns, which degrade image quality and confound conventional classification algorithms, including deep neural networks (DNNs). The impact of Moiré patterns remains largely unexplored for deepfake detectors. In this study, we investigate how camera-captured deepfake videos from digital screens affect detector performance. We conducted experiments using two prominent datasets, CelebDF and FF++, comparing the performance of four state-of-the-art detectors on camera-captured deepfake videos with introduced Moiré patterns. Our findings reveal a significant decline in detector accuracy, with none achieving above 68% on average. This underscores the critical need to address Moiré pattern challenges in real-world deepfake detection scenarios.

Exploring the Impact of Moire Pattern on Deepfake Detectors

TL;DR

This paper investigates how Moiré patterns, arising when camera-captured deepfakes are viewed on screens, degrade deepfake detectors deployed on external devices. Using camera-captured videos from CelebDF and FF++ and six state-of-the-art detectors, the study reveals substantial performance loss, with average F1-scores not exceeding 68% and some detectors performing as low as 47%. AUROC analyses show particularly weak discriminative power under Moiré artifacts, with CLRNet reaching only 0.64 on CelebDF. The work highlights the need for Moiré-aware preprocessing, diversified datasets, and cross-domain evaluation to enhance the real-world robustness of deepfake detection systems.

Abstract

Deepfake detection is critical in mitigating the societal threats posed by manipulated videos. While various algorithms have been developed for this purpose, challenges arise when detectors operate externally, such as on smartphones, when users take a photo of deepfake images and upload on the Internet. One significant challenge in such scenarios is the presence of Moiré patterns, which degrade image quality and confound conventional classification algorithms, including deep neural networks (DNNs). The impact of Moiré patterns remains largely unexplored for deepfake detectors. In this study, we investigate how camera-captured deepfake videos from digital screens affect detector performance. We conducted experiments using two prominent datasets, CelebDF and FF++, comparing the performance of four state-of-the-art detectors on camera-captured deepfake videos with introduced Moiré patterns. Our findings reveal a significant decline in detector accuracy, with none achieving above 68% on average. This underscores the critical need to address Moiré pattern challenges in real-world deepfake detection scenarios.
Paper Structure (24 sections, 4 figures, 1 table)

This paper contains 24 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Deepfake Detection
  4. Moiré Patterns and DNNs
  5. Evaluation Methodology
  6. Datasets for Experiments
  7. Videos Selection
  8. Moiré Pattern Introduction
  9. Camera-Captured Videos
  10. Experimental Setup
  11. Experiment Procedure
  12. Deepfake Detectors
  13. Evaluation Metrics
  14. Results
  15. Performance Degradation of SOTA detectors.
  16. ...and 9 more sections

Figures (4)

  • Figure 1: Comparison between an original frame (without Moiré pattern) and a camera-captured frame (with Moiré pattern). The top row displays a frame extracted from an authentic video, while the bottom row presents a frame from a deepfake video. The left column depicts the original frame (without the Moiré pattern), the center column demonstrates its appearance on a computer screen, and the right column provides a magnified view, clearly revealing the presence of the Moiré pattern.
  • Figure 2: The experimental setup involves the playback of videos sourced from FF++ and CelebDF on a computer screen. Subsequently, these videos are recorded using a smartphone camera. This method allows for the capture of the inherent Moiré pattern induced on the computer screen within the recorded video.
  • Figure 3: Average F1-score performance of five state-of-the-art (SOTA) deepfake detectors, when tested on camera-captured deepfake videos played on a computer screen, which introduce Moiré patterns. These videos are sourced from the FaceForensics++ and CelebDF datasets.
  • Figure 4: The AUROC (top) and Precision-Recall (bottom) curves of deepfake detectors against camera-captured deepfakes (containing Moiré patterns) from the CelebDF dataset.