Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AEROBLADE: Training-Free Detection of Latent Diffusion Images Using Autoencoder Reconstruction Error

Jonas Ricker, Denis Lukovnikov, Asja Fischer

TL;DR

This work tackles the rising challenge of visual disinformation from latent diffusion models by proposing AEROBLADE, a training-free detector that relies on autoencoder reconstruction error to distinguish real and generated images. By computing $Δ_{\text{AE}}(x)$ using the AE from multiple latent diffusion models and taking the minimum across them, AEROBLADE achieves high detection performance (AP approaching that of trained classifiers) across diverse models such as Stable Diffusion, Kandinsky, and Midjourney, while also providing qualitative localization of inpainted regions. The approach offers practical advantages in terms of simplicity, modularity, and robustness to perturbations, and it enables model-inventor disclosure without altering the backbone generators. The authors also analyze limitations, including dependence on access to AEs and potential biases in evaluation pipelines, and compare with existing baselines and DIRE-based methods to contextualize performance gains. Overall, AEROBLADE presents a compelling, training-free alternative for forensic analysis of LDM-generated imagery with actionable outputs for detection and localization.

Abstract

With recent text-to-image models, anyone can generate deceptively realistic images with arbitrary contents, fueling the growing threat of visual disinformation. A key enabler for generating high-resolution images with low computational cost has been the development of latent diffusion models (LDMs). In contrast to conventional diffusion models, LDMs perform the denoising process in the low-dimensional latent space of a pre-trained autoencoder (AE) instead of the high-dimensional image space. Despite their relevance, the forensic analysis of LDMs is still in its infancy. In this work we propose AEROBLADE, a novel detection method which exploits an inherent component of LDMs: the AE used to transform images between image and latent space. We find that generated images can be more accurately reconstructed by the AE than real images, allowing for a simple detection approach based on the reconstruction error. Most importantly, our method is easy to implement and does not require any training, yet nearly matches the performance of detectors that rely on extensive training. We empirically demonstrate that AEROBLADE is effective against state-of-the-art LDMs, including Stable Diffusion and Midjourney. Beyond detection, our approach allows for the qualitative analysis of images, which can be leveraged for identifying inpainted regions. We release our code and data at https://github.com/jonasricker/aeroblade .

AEROBLADE: Training-Free Detection of Latent Diffusion Images Using Autoencoder Reconstruction Error

TL;DR

This work tackles the rising challenge of visual disinformation from latent diffusion models by proposing AEROBLADE, a training-free detector that relies on autoencoder reconstruction error to distinguish real and generated images. By computing using the AE from multiple latent diffusion models and taking the minimum across them, AEROBLADE achieves high detection performance (AP approaching that of trained classifiers) across diverse models such as Stable Diffusion, Kandinsky, and Midjourney, while also providing qualitative localization of inpainted regions. The approach offers practical advantages in terms of simplicity, modularity, and robustness to perturbations, and it enables model-inventor disclosure without altering the backbone generators. The authors also analyze limitations, including dependence on access to AEs and potential biases in evaluation pipelines, and compare with existing baselines and DIRE-based methods to contextualize performance gains. Overall, AEROBLADE presents a compelling, training-free alternative for forensic analysis of LDM-generated imagery with actionable outputs for detection and localization.

Abstract

With recent text-to-image models, anyone can generate deceptively realistic images with arbitrary contents, fueling the growing threat of visual disinformation. A key enabler for generating high-resolution images with low computational cost has been the development of latent diffusion models (LDMs). In contrast to conventional diffusion models, LDMs perform the denoising process in the low-dimensional latent space of a pre-trained autoencoder (AE) instead of the high-dimensional image space. Despite their relevance, the forensic analysis of LDMs is still in its infancy. In this work we propose AEROBLADE, a novel detection method which exploits an inherent component of LDMs: the AE used to transform images between image and latent space. We find that generated images can be more accurately reconstructed by the AE than real images, allowing for a simple detection approach based on the reconstruction error. Most importantly, our method is easy to implement and does not require any training, yet nearly matches the performance of detectors that rely on extensive training. We empirically demonstrate that AEROBLADE is effective against state-of-the-art LDMs, including Stable Diffusion and Midjourney. Beyond detection, our approach allows for the qualitative analysis of images, which can be leveraged for identifying inpainted regions. We release our code and data at https://github.com/jonasricker/aeroblade .
Paper Structure (49 sections, 2 equations, 22 figures, 7 tables)

This paper contains 49 sections, 2 equations, 22 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Detection of Generated Images
  4. Diffusion Models for Visual Anomaly Detection
  5. Preliminaries
  6. Latent Diffusion Models (LDMs)
  7. Learned Perceptual Image Patch Similarity (LPIPS)
  8. Methodology
  9. Reconstruction-based Fake Image Detection
  10. AEROBLADE
  11. Experiments
  12. Setup
  13. Data
  14. Autoencoders
  15. Evaluation of Detection Performance
  16. ...and 34 more sections

Figures (22)

  • Figure 1: Example illustrating the idea behind AEROBLADE. (a) shows a real image from LAION-5B schuhmannLAION5BOpenLargescale2022 and (d) is generated by Stable Diffusion 2.1 rombachHighresolutionImageSynthesis2022. (b) and (e) are the corresponding reconstructions obtained by passing the original images through the AE of Stable Diffusion 2. (c) and (f) visualize the error between original and reconstruction measured using the LPIPS zhangUnreasonableEffectivenessDeep2018 distance. The reconstruction error is significantly lower for the generated image $x_\text{SD2.1}$ than for the real image $x_\text{real}$, which AEROBLADE leverages for detection.
  • Figure 2: Graphical representation of \ref{['eq:ae']}. The reconstruction error $\Delta_{\text{AE}_i}(x)$ is defined as the distance between an image $x$ and its reconstruction $\tilde{x}$ obtained from passing it through the encoder $\mathcal{E}_i$ and decoder $\mathcal{D}_i$ of an LDM's AE.
  • Figure 3: Distributions of reconstruction error $\Delta_\text{Min}$ using $\text{LPIPS}_2$ for different datasets. We provide results for all LPIPS variants in \ref{['sup:histograms']} in the supplementary material.
  • Figure 4: Example patches of size $128\times128$ with high (upper half) and low (lower half) reconstruction error. The reconstruction error is computed using $\text{LPIPS}_2$ and $\Delta_\text{Min}$, and the patches are selected from the top and bottom 1% of each dataset.
  • Figure 5: Reconstruction error against complexity for real and generated images. (b) contains generated images from all datasets, we provide individual plots in \ref{['sup:complexity']} in the supplementary material. The color map is clipped at 1000.0 samples for better visibility.
  • ...and 17 more figures