Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Static Artifacts: A Forensic Benchmark for Video Deepfake Reasoning in Vision Language Models

Zheyuan Gu, Qingsong Zhao, Yusong Wang, Zhaohong Huang, Xinqi Li, Cheng Yuan, Jiaowei Shao, Chi Zhang, Xuelong Li

TL;DR

Forensic Answer-Questioning (FAQ), a large-scale benchmark that formulates temporal deepfake analysis as a multiple-choice task, is proposed and a range of VLMs on FAQ are evaluated and a corresponding instruction-tuning set, FAQ-IT, is generated.

Abstract

Current Vision-Language Models (VLMs) for deepfake detection excel at identifying spatial artifacts but overlook a critical dimension: temporal inconsistencies in video forgeries. Adapting VLMs to reason about these dynamic cues remains a distinct challenge. To bridge this gap, we propose Forensic Answer-Questioning (FAQ), a large-scale benchmark that formulates temporal deepfake analysis as a multiple-choice task. FAQ introduces a three-level hierarchy to progressively evaluate and equip VLMs with forensic capabilities: (1) Facial Perception, testing the ability to identify static visual artifacts; (2) Temporal Deepfake Grounding, requiring the localization of dynamic forgery artifacts across frames; and (3) Forensic Reasoning, challenging models to synthesize evidence for final authenticity verdicts. We evaluate a range of VLMs on FAQ and generate a corresponding instruction-tuning set, FAQ-IT. Extensive experiments show that models fine-tuned on FAQ-IT achieve advanced performance on both in-domain and cross-dataset detection benchmarks. Ablation studies further validate the impact of our key design choices, confirming that FAQ is the driving force behind the temporal reasoning capabilities of these VLMs.

Beyond Static Artifacts: A Forensic Benchmark for Video Deepfake Reasoning in Vision Language Models

TL;DR

Forensic Answer-Questioning (FAQ), a large-scale benchmark that formulates temporal deepfake analysis as a multiple-choice task, is proposed and a range of VLMs on FAQ are evaluated and a corresponding instruction-tuning set, FAQ-IT, is generated.

Abstract

Current Vision-Language Models (VLMs) for deepfake detection excel at identifying spatial artifacts but overlook a critical dimension: temporal inconsistencies in video forgeries. Adapting VLMs to reason about these dynamic cues remains a distinct challenge. To bridge this gap, we propose Forensic Answer-Questioning (FAQ), a large-scale benchmark that formulates temporal deepfake analysis as a multiple-choice task. FAQ introduces a three-level hierarchy to progressively evaluate and equip VLMs with forensic capabilities: (1) Facial Perception, testing the ability to identify static visual artifacts; (2) Temporal Deepfake Grounding, requiring the localization of dynamic forgery artifacts across frames; and (3) Forensic Reasoning, challenging models to synthesize evidence for final authenticity verdicts. We evaluate a range of VLMs on FAQ and generate a corresponding instruction-tuning set, FAQ-IT. Extensive experiments show that models fine-tuned on FAQ-IT achieve advanced performance on both in-domain and cross-dataset detection benchmarks. Ablation studies further validate the impact of our key design choices, confirming that FAQ is the driving force behind the temporal reasoning capabilities of these VLMs.
Paper Structure (21 sections, 2 equations, 9 figures, 10 tables)

This paper contains 21 sections, 2 equations, 9 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional Deepfake Detection Datasets
  4. VLMs for Multimedia Forensics
  5. QA-based Deepfake Detection Datasets
  6. FAQ Benchmark
  7. Deepfake Video Curation
  8. Preprocessing
  9. Task Hierarchy in FAQ
  10. QA Generation with Forged Trace
  11. Human Verification
  12. FAQ Statistics
  13. Experiment
  14. Results
  15. Ablations
  16. ...and 6 more sections

Figures (9)

  • Figure 1: Illustration of our hierarchical benchmark.
  • Figure 2: An overview of FAQ's data construction pipeline.
  • Figure 3: Visualization of data distribution in FAQ across three levels. (a), (b) and (c) shows QA distribution under different facial region, duration and artifact, respectively.
  • Figure 4: Effect of frame sampling strategies.
  • Figure 5: Distribution of Videos across Face Detection Confidence Metrics and the Filtering Boundary. Each point represents a video from the original set. The filtering boundary (dashed light blue lines) separates the retained high-quality samples (blue points) from the removed low-quality samples (light purple points), demonstrating the rigorous exclusion of videos with unstable or weak face detections.
  • ...and 4 more figures