Mirage: Unveiling Hidden Artifacts in Synthetic Images with Large Vision-Language Models
Pranav Sharma, Shivank Garg, Durga Toshniwal
TL;DR
Mirage addresses the need for robust benchmarking of AI-generated image detectors in scenarios where subtle artifacts are present and many detectors fail. It introduces Mirage, a mid-scale dataset with 5,000 real and 5,000 artifact-bearing synthetic images, labeled via a nine-class artifact taxonomy and LVLM-assisted labeling, plus a real-world Chameleon benchmark for hard cases. The study shows LVLMs can outperform traditional detectors when artifacts are visible, but their accuracy drops on artifact-free fakes, while specialized detectors that leverage contrastive embeddings stay robust in such cases. It also demonstrates that LVLMs can provide interpretable, reasoning-based explanations, albeit sometimes at the cost of detection performance, highlighting a path for combining explainability with robust detection in future work.
Abstract
Recent advances in image generation models have led to models that produce synthetic images that are increasingly difficult for standard AI detectors to identify, even though they often remain distinguishable by humans. To identify this discrepancy, we introduce \textbf{Mirage}, a curated dataset comprising a diverse range of AI-generated images exhibiting visible artifacts, where current state-of-the-art detection methods largely fail. Furthermore, we investigate whether Large Vision-Language Models (LVLMs), which are increasingly employed as substitutes for human judgment in various tasks, can be leveraged for explainable AI image detection. Our experiments on both Mirage and existing benchmark datasets demonstrate that while LVLMs are highly effective at detecting AI-generated images with visible artifacts, their performance declines when confronted with images lacking such cues.