Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges

Ping Liu, Qiqi Tao, Joey Tianyi Zhou

TL;DR

The paper provides a comprehensive survey of facial deepfake detection, tracing the transition from single-modal to multi-modal approaches and highlighting the surge of diffusion-model–based forgeries. It offers a fine-grained taxonomy that separates passive from proactive methods and single-modal from multi-modal detection, and it covers state-of-the-art techniques leveraging Vision-Language Models and Multimodal Large Language Models. The work surveys datasets, evaluation protocols, and interpretability benchmarks, and it discusses proactive defenses tailored to emerging generators, including diffusion models. By identifying cross-cutting challenges such as generalization, multi-modal alignment, and the integration of large-scale multimodal models, the paper outlines practical directions for building robust, scalable, and explainable deepfake detection systems with real-world impact.

Abstract

As synthetic media, including video, audio, and text, become increasingly indistinguishable from real content, the risks of misinformation, identity fraud, and social manipulation escalate. This survey traces the evolution of deepfake detection from early single-modal methods to sophisticated multi-modal approaches that integrate audio-visual and text-visual cues. We present a structured taxonomy of detection techniques and analyze the transition from GAN-based to diffusion model-driven deepfakes, which introduce new challenges due to their heightened realism and robustness against detection. Unlike prior surveys that primarily focus on single-modal detection or earlier deepfake techniques, this work provides the most comprehensive study to date, encompassing the latest advancements in multi-modal deepfake detection, generalization challenges, proactive defense mechanisms, and emerging datasets specifically designed to support new interpretability and reasoning tasks. We further explore the role of Vision-Language Models (VLMs) and Multimodal Large Language Models (MLLMs) in strengthening detection robustness against increasingly sophisticated deepfake attacks. By systematically categorizing existing methods and identifying emerging research directions, this survey serves as a foundation for future advancements in combating AI-generated facial forgeries. A curated list of all related papers can be found at \href{https://github.com/qiqitao77/Comprehensive-Advances-in-Deepfake-Detection-Spanning-Diverse-Modalities}{https://github.com/qiqitao77/Awesome-Comprehensive-Deepfake-Detection}.

Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges

TL;DR

The paper provides a comprehensive survey of facial deepfake detection, tracing the transition from single-modal to multi-modal approaches and highlighting the surge of diffusion-model–based forgeries. It offers a fine-grained taxonomy that separates passive from proactive methods and single-modal from multi-modal detection, and it covers state-of-the-art techniques leveraging Vision-Language Models and Multimodal Large Language Models. The work surveys datasets, evaluation protocols, and interpretability benchmarks, and it discusses proactive defenses tailored to emerging generators, including diffusion models. By identifying cross-cutting challenges such as generalization, multi-modal alignment, and the integration of large-scale multimodal models, the paper outlines practical directions for building robust, scalable, and explainable deepfake detection systems with real-world impact.

Abstract

As synthetic media, including video, audio, and text, become increasingly indistinguishable from real content, the risks of misinformation, identity fraud, and social manipulation escalate. This survey traces the evolution of deepfake detection from early single-modal methods to sophisticated multi-modal approaches that integrate audio-visual and text-visual cues. We present a structured taxonomy of detection techniques and analyze the transition from GAN-based to diffusion model-driven deepfakes, which introduce new challenges due to their heightened realism and robustness against detection. Unlike prior surveys that primarily focus on single-modal detection or earlier deepfake techniques, this work provides the most comprehensive study to date, encompassing the latest advancements in multi-modal deepfake detection, generalization challenges, proactive defense mechanisms, and emerging datasets specifically designed to support new interpretability and reasoning tasks. We further explore the role of Vision-Language Models (VLMs) and Multimodal Large Language Models (MLLMs) in strengthening detection robustness against increasingly sophisticated deepfake attacks. By systematically categorizing existing methods and identifying emerging research directions, this survey serves as a foundation for future advancements in combating AI-generated facial forgeries. A curated list of all related papers can be found at \href{https://github.com/qiqitao77/Comprehensive-Advances-in-Deepfake-Detection-Spanning-Diverse-Modalities}{https://github.com/qiqitao77/Awesome-Comprehensive-Deepfake-Detection}.
Paper Structure (33 sections, 3 equations, 5 figures, 2 tables)

This paper contains 33 sections, 3 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Task Formulation for Facial Forgery Detection
  4. Datasets for Facial Forgery Detection
  5. Single-modal (visual) Deepfake Datasets
  6. GANs/VAEs based Generated Datasets.
  7. DMs based Generated Datasets.
  8. Interpretability-oriented Benchmark
  9. Multi-modal Deepfake Datasets
  10. Audio-Visual Deepfake Datasets
  11. Text-Visual Deepfake Datasets
  12. Evaluation
  13. History and Challenges of Deepfake Datasets
  14. Method
  15. Single-modal Detection
  16. ...and 18 more sections

Figures (5)

  • Figure 1: Advances in facial deepfake generation and corresponding detection in recent years.
  • Figure 2: Multi-modal deepfake detection.
  • Figure 3: The taxonomy of facial deepfake detection methods.
  • Figure 4: The comparison between passive and proactive deepfake detection methods. Figure from wang2022anti_ijcai2022.
  • Figure 5: Three main paradigms for multi-modal audio-visual deepfake detection.