FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models

TL;DR

FakeBench addresses the explainability gap in fake image detection by introducing a multimodal benchmark that probes large multimodal models across detection, interpretation, reasoning, and fine-grained forgery analysis. It builds three data ingredients—FakeClass, FakeClue, and FakeQA—and uses human-in-the-loop annotations to produce authentic forgery cues. Experimental results show that some LMMs reach near-human performance in detection but lag in reasoning and forgery analysis, and chain-of-thought prompting often fails to improve performance. The work highlights the need for authenticity-focused data and for integrating forensic knowledge into LMMs to enable transparent and trustworthy fake image detection.

Abstract

The ability to distinguish whether an image is generated by artificial intelligence (AI) is a crucial ingredient in human intelligence, usually accompanied by a complex and dialectical forensic and reasoning process. However, current fake image detection models and databases focus on binary classification without understandable explanations for the general populace. This weakens the credibility of authenticity judgment and may conceal potential model biases. Meanwhile, large multimodal models (LMMs) have exhibited immense visual-text capabilities on various tasks, bringing the potential for explainable fake image detection. Therefore, we pioneer the probe of LMMs for explainable fake image detection by presenting a multimodal database encompassing textual authenticity descriptions, the FakeBench. For construction, we first introduce a fine-grained taxonomy of generative visual forgery concerning human perception, based on which we collect forgery descriptions in human natural language with a human-in-the-loop strategy. FakeBench examines LMMs with four evaluation criteria: detection, reasoning, interpretation and fine-grained forgery analysis, to obtain deeper insights into image authenticity-relevant capabilities. Experiments on various LMMs confirm their merits and demerits in different aspects of fake image detection tasks. This research presents a paradigm shift towards transparency for the fake image detection area and reveals the need for greater emphasis on forensic elements in visual-language research and AI risk control. FakeBench will be available at https://github.com/Yixuan423/FakeBench.

Figures (7)

• Figure 1: Scope of research questions with exemplar question-answer pairs from the proposed FakeBench dataset. (a) Fake image detection; (b) fake image reasoning; (c) fake image interpretation; (d) fine-grained forgery analyses. The responses are exemplars provided by humans.
• Figure 2: Examples and content breakdown for the fake images in FakeBench. In total, ten different deep generative models are incorporated with balanced content.
• Figure 3: Subjective analyses on image authenticity judgment. (a) Human distinguishing analyses regarding accuracy on genuine and fake images. The left part illustrates the estimated distributions for the accuracy of each authenticity category, and the detailed statistics are on the right side. (b) The proposed taxonomy of authenticity-relevant judging criteria and human-rating statistics. Different human subjects contribute to each subset's best and worst performances.
• Figure 4: Prompt sheet of the proposed FakeBench, including: (a) basic mode for FakeClass, (b) fault-finding mode based on in-context prompting for FakeClue, (c) inference mode based on CoT prompting for FakeClue, and (d) free mode for FakeQA.
• Figure 5: Human-in-the-loop strategy for producing the golden clues. Human and GPT-4V work in collaboration to ensure the quality of descriptions.