ForensicHub: A Unified Benchmark & Codebase for All-Domain Fake Image Detection and Localization

TL;DR

ForensicHub tackles the fragmentation of fake image detection and localization by introducing the first all-domain benchmark and codebase for cross-domain evaluation. It implements a modular, adapter-driven framework with a configurable workflow and an Image Forensic Fusion Protocol (IFF-Protocol) to train with balanced multi-domain data and evaluate across unseen domains without fine-tuning. The benchmark unifies four domains (Deepfake, IMDL, AIGC, Document), provides 23 datasets, 42 models, 6 backbones, and 11 metrics, and demonstrates cross-domain transferability and generalization patterns, offering eight practical insights for future design. This work advances reproducibility, cross-task comparability, and the development of robust, generalizable FIDL methods with broad societal impact.

Abstract

The field of Fake Image Detection and Localization (FIDL) is highly fragmented, encompassing four domains: deepfake detection (Deepfake), image manipulation detection and localization (IMDL), artificial intelligence-generated image detection (AIGC), and document image manipulation localization (Doc). Although individual benchmarks exist in some domains, a unified benchmark for all domains in FIDL remains blank. The absence of a unified benchmark results in significant domain silos, where each domain independently constructs its datasets, models, and evaluation protocols without interoperability, preventing cross-domain comparisons and hindering the development of the entire FIDL field. To close the domain silo barrier, we propose ForensicHub, the first unified benchmark & codebase for all-domain fake image detection and localization. Considering drastic variations on dataset, model, and evaluation configurations across all domains, as well as the scarcity of open-sourced baseline models and the lack of individual benchmarks in some domains, ForensicHub: i) proposes a modular and configuration-driven architecture that decomposes forensic pipelines into interchangeable components across datasets, transforms, models, and evaluators, allowing flexible composition across all domains; ii) fully implements 10 baseline models, 6 backbones, 2 new benchmarks for AIGC and Doc, and integrates 2 existing benchmarks of DeepfakeBench and IMDLBenCo through an adapter-based design; iii) conducts indepth analysis based on the ForensicHub, offering 8 key actionable insights into FIDL model architecture, dataset characteristics, and evaluation standards. ForensicHub represents a significant leap forward in breaking the domain silos in the FIDL field and inspiring future breakthroughs.

Figures (2)

• Figure 1: Overview of our ForensicHub. It is compatible with DeepfakeBench and IMDLBenCo via adapters, and introduces new AIGC and Document benchmarks. ForensicHub allows datasets and models from any domain to be freely combined into custom pipelines.
• Figure 2: Grad-CAM visualization (zoomed in for better visualization).