Towards Real-World Deepfake Detection: A Diverse In-the-wild Dataset of Forgery Faces

TL;DR

This work tackles the mismatch between academic deepfake benchmarks and real-world forgery scenarios by introducing RedFace, a large real-world oriented dataset with over 60k forged images and 1k manipulated videos generated via nine commercial online platforms. The dataset covers four manipulation types—Entire Face Synthesis, Face Swapping, Face Attribute Manipulation, and Face Reenactment—enabled by diverse generation tools and accompanied by detailed operation annotations. Extensive cross-domain and in-domain evaluations show that leading detectors trained on standard benchmarks struggle to generalize to RedFace, especially under real-world degradations, underscoring the need for more robust, generalizable detection approaches. The work provides critical insights into how real-world deepfake content differs from academic data and motivates the development of detectors resilient to black-box generation and platform-driven diversity, with practical implications for social media integrity and digital forensics.

Abstract

Deepfakes, leveraging advanced AIGC (Artificial Intelligence-Generated Content) techniques, create hyper-realistic synthetic images and videos of human faces, posing a significant threat to the authenticity of social media. While this real-world threat is increasingly prevalent, existing academic evaluations and benchmarks for detecting deepfake forgery often fall short to achieve effective application for their lack of specificity, limited deepfake diversity, restricted manipulation techniques.To address these limitations, we introduce RedFace (Real-world-oriented Deepfake Face), a specialized facial deepfake dataset, comprising over 60,000 forged images and 1,000 manipulated videos derived from authentic facial features, to bridge the gap between academic evaluations and real-world necessity. Unlike prior benchmarks, which typically rely on academic methods to generate deepfakes, RedFace utilizes 9 commercial online platforms to integrate the latest deepfake technologies found "in the wild", effectively simulating real-world black-box scenarios.Moreover, RedFace's deepfakes are synthesized using bespoke algorithms, allowing it to capture diverse and evolving methods used by real-world deepfake creators. Extensive experimental results on RedFace (including cross-domain, intra-domain, and real-world social network dissemination simulations) verify the limited practicality of existing deepfake detection schemes against real-world applications. We further perform a detailed analysis of the RedFace dataset, elucidating the reason of its impact on detection performance compared to conventional datasets. Our dataset is available at: https://github.com/kikyou-220/RedFace.

Figures (5)

• Figure 1: An overview of our benchmark scheme
• Figure 2: Word frequency distribution in face synthesis descriptive terms
• Figure 3: Visualization of images of RedFace dataset
• Figure 4: Process of our deepfake generation
• Figure 5: Cross Forgery Detection