Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI-Generated Image Detectors Overrely on Global Artifacts: Evidence from Inpainting Exchange

Elif Nebioglu, Emirhan Bilgiç, Adrian Popescu

TL;DR

The paper reveals that AI-generated image detectors predominantly exploit global artifacts created by VAE-based inpainting, rather than detecting locally synthesized content. It introduces Inpainting Exchange (INP-X), which restores background pixels outside the edited region to remove global cues, and provides a 90K-image benchmark to quantify detector reliance on these artifacts. Theoretical analysis links spectral attenuation to VAE bottlenecks, and empirical results show pretrained detectors—and even commercial APIs—drop from about 0.91 accuracy to around 0.55 under INP-X, highlighting a robustness gap. Training detectors on INP-X improves generalization and localization, underscoring the need for content-aware detection methods and more robust evaluation frameworks that account for realistic post-edits.

Abstract

Modern deep learning-based inpainting enables realistic local image manipulation, raising critical challenges for reliable detection. However, we observe that current detectors primarily rely on global artifacts that appear as inpainting side effects, rather than on locally synthesized content. We show that this behavior occurs because VAE-based reconstruction induces a subtle but pervasive spectral shift across the entire image, including unedited regions. To isolate this effect, we introduce Inpainting Exchange (INP-X), an operation that restores original pixels outside the edited region while preserving all synthesized content. We create a 90K test dataset including real, inpainted, and exchanged images to evaluate this phenomenon. Under this intervention, pretrained state-of-the-art detectors, including commercial ones, exhibit a dramatic drop in accuracy (e.g., from 91\% to 55\%), frequently approaching chance level. We provide a theoretical analysis linking this behavior to high-frequency attenuation caused by VAE information bottlenecks. Our findings highlight the need for content-aware detection. Indeed, training on our dataset yields better generalization and localization than standard inpainting. Our dataset and code are publicly available at https://github.com/emirhanbilgic/INP-X.

AI-Generated Image Detectors Overrely on Global Artifacts: Evidence from Inpainting Exchange

TL;DR

The paper reveals that AI-generated image detectors predominantly exploit global artifacts created by VAE-based inpainting, rather than detecting locally synthesized content. It introduces Inpainting Exchange (INP-X), which restores background pixels outside the edited region to remove global cues, and provides a 90K-image benchmark to quantify detector reliance on these artifacts. Theoretical analysis links spectral attenuation to VAE bottlenecks, and empirical results show pretrained detectors—and even commercial APIs—drop from about 0.91 accuracy to around 0.55 under INP-X, highlighting a robustness gap. Training detectors on INP-X improves generalization and localization, underscoring the need for content-aware detection methods and more robust evaluation frameworks that account for realistic post-edits.

Abstract

Modern deep learning-based inpainting enables realistic local image manipulation, raising critical challenges for reliable detection. However, we observe that current detectors primarily rely on global artifacts that appear as inpainting side effects, rather than on locally synthesized content. We show that this behavior occurs because VAE-based reconstruction induces a subtle but pervasive spectral shift across the entire image, including unedited regions. To isolate this effect, we introduce Inpainting Exchange (INP-X), an operation that restores original pixels outside the edited region while preserving all synthesized content. We create a 90K test dataset including real, inpainted, and exchanged images to evaluate this phenomenon. Under this intervention, pretrained state-of-the-art detectors, including commercial ones, exhibit a dramatic drop in accuracy (e.g., from 91\% to 55\%), frequently approaching chance level. We provide a theoretical analysis linking this behavior to high-frequency attenuation caused by VAE information bottlenecks. Our findings highlight the need for content-aware detection. Indeed, training on our dataset yields better generalization and localization than standard inpainting. Our dataset and code are publicly available at https://github.com/emirhanbilgic/INP-X.
Paper Structure (45 sections, 4 theorems, 17 equations, 17 figures, 6 tables)

This paper contains 45 sections, 4 theorems, 17 equations, 17 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. AI-Generated Image Detection.
  4. Universal Detectors and Generalization.
  5. Shortcut Learning and Dataset Biases.
  6. Manipulation, Localization, and Forensic Analysis.
  7. Adversarial and Anti-Forensic Methods.
  8. Benchmarks for Manipulation Detection.
  9. Theoretical Analysis
  10. Latent Diffusion Inpainting
  11. Spectral Bias in VAE Reconstruction
  12. Empirical Validation: VAE as the Source of Artifacts.
  13. INP-X and Divergence Minimization
  14. Experiments and Analysis
  15. Dataset Construction
  16. ...and 30 more sections

Key Result

Theorem 3.2

Let $\mathcal{T}$ be an autoencoder trained to minimize a reconstruction objective dominated by the $L_2$ norm (MSE), i.e., $\mathcal{L} = \|x - \mathcal{T}(x)\|^2$. Assuming the latent code $z = \mathcal{E}(x)$ effectively captures the semantic signal $s$ but is approximately independent of the sto where $\Omega_{high}$ denotes the frequency band dominated by sensor noise $n$.

Figures (17)

  • Figure 1: a) Overview of the INP-X pipeline. The unmasked regions of the generated output are replaced with the corresponding regions from the original image, preserving original content while retaining the synthesized masked area. b) The difference between Original-Inpainted and Original-Ours. c) INP: Inpainting Pipeline.
  • Figure 3: Verification of VAE-induced artifacts. Columns from left to right: (1) Original Image, (2) Mask, (3) Inpainted Result, (4) High-Frequency Filters of the Original Image, (5) VAE Reconstruction Artifacts (decoding encoded original), (6) Difference Map (Inpainted - Original). Note the strong structural correlation between the VAE artifacts and the final inpainting difference, confirming the VAE as the primary source of global noise.
  • Figure 4: Scatter plots showing strong correlations between VAE reconstruction loss and inpainting artifacts.
  • Figure 5: Impact of mask coverage on detection accuracy. Larger masks generally enable better detection, but our exchange method consistently degrades performance compared to standard inpainting across all sizes.
  • Figure 6: Sequence of correlation matrices showing the relationships between different error metrics across datasets.
  • ...and 12 more figures

Theorems & Definitions (12)

  • Definition 3.1: Spectral Variance Gap
  • Theorem 3.2: Variance Contraction in VAEs
  • proof
  • Remark 3.3: Extension to Modern VAEs
  • Theorem 3.4: Divergence Reduction via Exchange
  • proof
  • Definition 1.1: Multiresolution Approximation mallat1989multiresolution
  • Theorem 1.2: Wavelet attenuation under spatial compression
  • proof : Sketch
  • Corollary 1.3: Detectability via wavelet modulus maxima
  • ...and 2 more