LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models

TL;DR

LOKI introduces a comprehensive multimodal benchmark to evaluate synthetic data detection by large multimodal models across video, image, 3D, text, and audio. It provides 18K questions over 26 subcategories with multi-level annotations and anomaly-explanation tasks, enabling zero-shot assessment of model perception, reasoning, and explainability. Key findings show LMMs offer умерate detection capability with strong explanatory potential but suffer from biases and domain-specific knowledge gaps, especially in audio and specialized imagery. The benchmark serves as a rigorous platform to drive the development of more balanced, interpretable synthetic data detectors and to probe the limits of current LMMs.

Abstract

With the rapid development of AI-generated content, the future internet may be inundated with synthetic data, making the discrimination of authentic and credible multimodal data increasingly challenging. Synthetic data detection has thus garnered widespread attention, and the performance of large multimodal models (LMMs) in this task has attracted significant interest. LMMs can provide natural language explanations for their authenticity judgments, enhancing the explainability of synthetic content detection. Simultaneously, the task of distinguishing between real and synthetic data effectively tests the perception, knowledge, and reasoning capabilities of LMMs. In response, we introduce LOKI, a novel benchmark designed to evaluate the ability of LMMs to detect synthetic data across multiple modalities. LOKI encompasses video, image, 3D, text, and audio modalities, comprising 18K carefully curated questions across 26 subcategories with clear difficulty levels. The benchmark includes coarse-grained judgment and multiple-choice questions, as well as fine-grained anomaly selection and explanation tasks, allowing for a comprehensive analysis of LMMs. We evaluated 22 open-source LMMs and 6 closed-source models on LOKI, highlighting their potential as synthetic data detectors and also revealing some limitations in the development of LMM capabilities. More information about LOKI can be found at https://opendatalab.github.io/LOKI/

Figures (63)

• Figure 1: Overview of LOKI benchmark. LOKI possesses four key characteristics: 1) Diverse modalities (video, image, 3D, text and audio); 2) Heterogeneous categories (26 detailed subcategories); 3) Multi-level annotations; 4) Multimodal synthetic data evaluation framework.
• Figure 2: Statistical information of LOKI. The left side displays the detailed categories of each modality, while the right side presents the questions across different modalities. The inner circle numbers represent the data volume, and the outer circle numbers indicate the number of questions.
• Figure 3: Examples of Synthetic Data Annotations: (a) Detailed annotations of video anomalies; (b) Detailed annotations of image anomalies; (c) Detailed annotations of 3D anomalies.
• Figure 4: Example Questions of LOKI. LOKI includes four types of questions:(a) Judgment questions; (b) Multiple choice questions; (c) Abnormal detail selection; (d) Abnormal explanation.
• Figure 5: The multimodal large model capability assessment analysis results. (a) Model bias assessment, where the closer the color is to red, the more the model is biased towards classifying the data as real; the closer to blue, the more it leans towards synthetic data. The size of the square also represents the degree of bias. (b) The performance of GPT-4o across different image types and its difference from human users. (c) A relative radar chart of the model's performance across various modalities, with Human benchmarks for comparison.