SimLBR: Learning to Detect Fake Images by Learning to Detect Real Images

TL;DR

This work argues that it is more principled to learn a tight decision boundary around the real image distribution and treat the fake category as a sink class, and proposes SimLBR, a simple and efficient framework for fake image detection using Latent Blending Regularization (LBR).

Abstract

The rapid advancement of generative models has made the detection of AI-generated images a critical challenge for both research and society. Recent works have shown that most state-of-the-art fake image detection methods overfit to their training data and catastrophically fail when evaluated on curated hard test sets with strong distribution shifts. In this work, we argue that it is more principled to learn a tight decision boundary around the real image distribution and treat the fake category as a sink class. To this end, we propose SimLBR, a simple and efficient framework for fake image detection using Latent Blending Regularization (LBR). Our method significantly improves cross-generator generalization, achieving up to +24.85\% accuracy and +69.62\% recall on the challenging Chameleon benchmark. SimLBR is also highly efficient, training orders of magnitude faster than existing approaches. Furthermore, we emphasize the need for reliability-oriented evaluation in fake image detection, introducing risk-adjusted metrics and worst-case estimates to better assess model robustness. All code and models will be released on HuggingFace and GitHub.

Figures (9)

• Figure 1: t-SNE Visualization of Latent Blending Regularization: Prior methods tend to overfit to generator-specific artifacts, leading to fake images from unseen models being misclassified as real. Incorporating LBR encourages the detector to learn a tighter boundary around the real image distribution (blue/orange cluster), ensuring that any sample lying outside this region—regardless of its generative source—is correctly identified as fake.
• Figure 2: Framework of SimLBR: Our model samples either a real or a fake label for a real image in the training set. If a fake label is sampled, we blend small amounts of fake image information in a pretrained latent space. Solving this objective forces the model to learn a tighter decision boundary around the unperturbed real image distribution enabling better generalization to unseen generative models.
• Figure 3: (a) Ablation of $\alpha$: We train SimLBR using ProGAN and different sampling distributions for $\alpha$. Always retaining at least half the information from the real image is one of the optimal and robust choices for sampling $\alpha$. (b) Ablation of MLP: We try different sizes of MLP. SimLBR performs better with smaller MLPs and overfits with large number of hidden layers.
• Figure 4: Chameleon
• Figure 5: Wukong