AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Models

TL;DR

This work tackles the dual challenge of explainability and generalization in AI-generated image detection amid rapidly evolving AIGC. It introduces Holmes-Set (Holmes-SFTSet and Holmes-DPOSet) constructed via a Multi-Expert Jury to provide human-verifiable explanations and human-aligned preferences, addressing training-data scarcity and alignment gaps. The Holmes Pipeline combines Visual Expert Pre-training, supervised fine-tuning, and direct preference optimization to adapt Multimodal Large Language Models for detection, paired with a collaborative decoding strategy at inference to improve generalization. Empirical results show state-of-the-art detection accuracy on multiple benchmarks and superior explanation quality compared to existing MLLMs, highlighting the practical impact for trustworthy AIGI detection in dynamic real-world scenarios.

Abstract

The rapid development of AI-generated content (AIGC) technology has led to the misuse of highly realistic AI-generated images (AIGI) in spreading misinformation, posing a threat to public information security. Although existing AIGI detection techniques are generally effective, they face two issues: 1) a lack of human-verifiable explanations, and 2) a lack of generalization in the latest generation technology. To address these issues, we introduce a large-scale and comprehensive dataset, Holmes-Set, which includes the Holmes-SFTSet, an instruction-tuning dataset with explanations on whether images are AI-generated, and the Holmes-DPOSet, a human-aligned preference dataset. Our work introduces an efficient data annotation method called the Multi-Expert Jury, enhancing data generation through structured MLLM explanations and quality control via cross-model evaluation, expert defect filtering, and human preference modification. In addition, we propose Holmes Pipeline, a meticulously designed three-stage training framework comprising visual expert pre-training, supervised fine-tuning, and direct preference optimization. Holmes Pipeline adapts multimodal large language models (MLLMs) for AIGI detection while generating human-verifiable and human-aligned explanations, ultimately yielding our model AIGI-Holmes. During the inference stage, we introduce a collaborative decoding strategy that integrates the model perception of the visual expert with the semantic reasoning of MLLMs, further enhancing the generalization capabilities. Extensive experiments on three benchmarks validate the effectiveness of our AIGI-Holmes.

Figures (30)

• Figure 1: (a): Comparison of AIGI-Holmes with existing methods, (b): A qualitative example to illustrate the effect of AIGI-Holmes, (c): AIGI-Holmes outperforms existing baseline methods on state-of-the-art generators under unseen settings.
• Figure 2: Details of the Holmes-Set Construction. The figure illustrates our data pipeline, consisting of four key components: Data Source (including Data Collection and Image Generation), Automated Annotation, Preference Modification (based on human expert feedback), and Comprehensive Evaluation (to assess model generalizability and interpretability).
• Figure 3: Overview of AIGI-Holmes. We enhance LLaVA liu2024visual with NPR tan2024rethinking visual expert $\mathcal{R}$ and the Holmes Pipeline, featuring three training stages: Visual Expert Pre-training, SFT, DPO, and a collaborative decoding strategy during inference.
• Figure 4: Qualitative results of AIGI-Holmes on AI-Generated images.
• Figure 5: Robustness of the explanation on JPEG Compression (QF=70), Gaussian Blur ($\sigma=2$), and Resize ($\times 0.5$) of AIGI-Holmes.