NeXT-IMDL: Build Benchmark for NeXT-Generation Image Manipulation Detection & Localization

TL;DR

NeXT-IMDL tackles the generalization gaps in image manipulation detection and localization by introducing a diagnostic benchmark that organizes AIGC-based manipulations along four axes: editing models, manipulation types, content semantics, and forgery granularity. It pairs this framework with five cross-dimension evaluation protocols and a large, diverse dataset built from 32 editing tools and 558,269 manipulated samples to stress-test detectors. Across 11 representative models, the study reveals significant brittleness under cross-setting evaluations, including semantic and granularity shifts, removal-based pre-training advantages for cross-task generalization, and a dominant performance of MaskCLIP that leverages foundation-model fusion. These findings underscore the limitations of current IMDL approaches and establish NeXT-IMDL as a crucial benchmark to drive the development of robust, generalizable detectors in the AIGC era.

Abstract

The accessibility surge and abuse risks of user-friendly image editing models have created an urgent need for generalizable, up-to-date methods for Image Manipulation Detection and Localization (IMDL). Current IMDL research typically uses cross-dataset evaluation, where models trained on one benchmark are tested on others. However, this simplified evaluation approach conceals the fragility of existing methods when handling diverse AI-generated content, leading to misleading impressions of progress. This paper challenges this illusion by proposing NeXT-IMDL, a large-scale diagnostic benchmark designed not just to collect data, but to probe the generalization boundaries of current detectors systematically. Specifically, NeXT-IMDL categorizes AIGC-based manipulations along four fundamental axes: editing models, manipulation types, content semantics, and forgery granularity. Built upon this, NeXT-IMDL implements five rigorous cross-dimension evaluation protocols. Our extensive experiments on 11 representative models reveal a critical insight: while these models perform well in their original settings, they exhibit systemic failures and significant performance degradation when evaluated under our designed protocols that simulate real-world, various generalization scenarios. By providing this diagnostic toolkit and the new findings, we aim to advance the development towards building truly robust, next-generation IMDL models.

Figures (26)

• Figure 1: Overview of NeXT-IMDL. We organize predominant works in various dimensions. Moreover, we propose to decouple the AIGC IMDL task into four key aspects and build five comprehensive evaluation protocols. Insights are extracted from the results of our extensive experiments.
• Figure 2: Generation pipeline of our proposed NeXT-IMDL dataset.
• Figure 3: Samples from IMDL.
• Figure 4: A brief visualization of models' IoU scores in different protocols. It can be observed from the chart that models' performances would decline to varying degrees in all four cross-scene settings.
• Figure 5: Qualitative results on NeXT-IMDL, Protocol-1.