DDL: A Large-Scale Datasets for Deepfake Detection and Localization in Diversified Real-World Scenarios

TL;DR

DDL addresses the interpretability and localization gap in deepfake benchmarks by offering a large-scale, multi-modal dataset with fine-grained ground truth. It combines a unified LLM- and human-driven generation pipeline to produce over $1.4\mathrm{M}+$ forged samples across $80$ methods, together with $1.18\mathrm{M}+$ spatial masks and $0.23\mathrm{M}+$ temporal segments. The dataset spans image, audio, and video modalities with diverse forgery scenarios and modalities, and includes out-of-distribution test sets to mirror real-world conditions. Empirical benchmarks demonstrate the dataset’s utility for advancing next-generation deepfake detection, localization, and interpretability, including practical deployment validation on an AIGC detection platform yielding high accuracy in diverse settings.

Abstract

Recent advances in AIGC have exacerbated the misuse of malicious deepfake content, making the development of reliable deepfake detection methods an essential means to address this challenge. Although existing deepfake detection models demonstrate outstanding performance in detection metrics, most methods only provide simple binary classification results, lacking interpretability. Recent studies have attempted to enhance the interpretability of classification results by providing spatial manipulation masks or temporal forgery segments. However, due to the limitations of forgery datasets, the practical effectiveness of these methods remains suboptimal. The primary reason lies in the fact that most existing deepfake datasets contain only binary labels, with limited variety in forgery scenarios, insufficient diversity in deepfake types, and relatively small data scales, making them inadequate for complex real-world scenarios.To address this predicament, we construct a novel large-scale deepfake detection and localization (\textbf{DDL}) dataset containing over $\textbf{1.4M+}$ forged samples and encompassing up to $\textbf{80}$ distinct deepfake methods. The DDL design incorporates four key innovations: (1) \textbf{Comprehensive Deepfake Methods} (covering 7 different generation architectures and a total of 80 methods), (2) \textbf{Varied Manipulation Modes} (incorporating 7 classic and 3 novel forgery modes), (3) \textbf{Diverse Forgery Scenarios and Modalities} (including 3 scenarios and 3 modalities), and (4) \textbf{Fine-grained Forgery Annotations} (providing 1.18M+ precise spatial masks and 0.23M+ precise temporal segments).Through these improvements, our DDL not only provides a more challenging benchmark for complex real-world forgeries but also offers crucial support for building next-generation deepfake detection, localization, and interpretability methods.

Figures (6)

• Figure 1: Overview of our DDL dataset. It shows the DDL's advantages in comprehensive deepfake methods, varied manipulation modes, diverse forgery scenarios and modalities, and fine-grained forgery annotations.
• Figure 2: The LLM- and human-driven deepfake generation pipeline comprises four key components: Source Data Collection (including real face data from 14 distinct sources), Deepfake Generation (covering generative models across four different scenarios), Human-in-the-Loop Oversight (encompassing generation prompt curation and deepfake sample quality and annotation).
• Figure 3: Examples of Hybrid Facial Forgery (HFF) Mode.
• Figure 4: Examples of Audio-Visual Full Synthesis (AVFS) Mode.
• Figure 5: Examples of Audio-Visual Asynchronous Manipulation (AVAM) Mode.