Detecting Localized Deepfakes: How Well Do Synthetic Image Detectors Handle Inpainting?

TL;DR

This work benchmarks whether state-of-the-art detectors trained on fully synthetic images can generalize to localized inpainting and region-level edits, a growing threat in cybersecurity-relevant media. Using two backbone families (CNN-based ResNet-50 CLIP and self-supervised vision transformers) and AI-GenBench training, the authors evaluate transfer performance across BR-Gen, TGIF, and TGIF2, varying mask sizes, inpainting types, and compression. The results show partial transferability: detectors perform best on medium-to-large edits and full-regeneration scenarios, with significant degradation on small or subtle inpaintings, highlighting the limitations of binary real-vs-fake classification for localization. The findings emphasize the need for hybrid detection systems that couple global classifiers with localized, segmentation-aware cues to improve robustness against localized deepfakes.

Abstract

The rapid progress of generative AI has enabled highly realistic image manipulations, including inpainting and region-level editing. These approaches preserve most of the original visual context and are increasingly exploited in cybersecurity-relevant threat scenarios. While numerous detectors have been proposed for identifying fully synthetic images, their ability to generalize to localized manipulations remains insufficiently characterized. This work presents a systematic evaluation of state-of-the-art detectors, originally trained for the deepfake detection on fully synthetic images, when applied to a distinct challenge: localized inpainting detection. The study leverages multiple datasets spanning diverse generators, mask sizes, and inpainting techniques. Our experiments show that models trained on a large set of generators exhibit partial transferability to inpainting-based edits and can reliably detect medium- and large-area manipulations or regeneration-style inpainting, outperforming many existing ad hoc detection approaches.

Figures (5)

• Figure 1: Inpainted images from different generators.
• Figure 2: Mask size distribution across the datasets.
• Figure 3: Distribution of masked area percentages across representative generative models. For clarity and compactness, models with identical distributions in terms of number of images per bin have been grouped together, and only one representative model from each group is shown.
• Figure 4: AUROC as a function of mask size (percentage of image area modified). Larger regions yield more detectable artifacts.
• Figure 5: Qualitative examples of inpainted images with corresponding masks, illustrating successful detections (left) and failure cases (right) for the DINOv2 model.