Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Binary Classification: A Semi-supervised Approach to Generalized AI-generated Image Detection

Hong-Hanh Nguyen-Le, Van-Tuan Tran, Dinh-Thuc Nguyen, Nhien-An Le-Khac

TL;DR

This work addresses the challenge of detecting AI-generated images with limited cross-generator generalization, especially across GANs and diffusion models. It provides a theoretical analysis showing that GANs optimize the Jensen-Shannon divergence while diffusion models minimize a KL-divergence bound, leading to partial vs complete data-manifold coverage and distinct latent artifacts. Based on these insights, the authors propose TriDetect, a semi-supervised detector that jointly performs real/fake classification and latent-architecture pattern discovery using balanced optimal-transport clustering (via Sinkhorn-Knopp) and cross-view consistency. Extensive experiments across GenImage, AIGCDetectBenchmark, and in-the-wild datasets demonstrate TriDetect’s strong generalization to unseen generators within the same architectural families and robustness to degradations, highlighting a new architecture-centered paradigm for AI-generated image detection.

Abstract

The rapid advancement of generators (e.g., StyleGAN, Midjourney, DALL-E) has produced highly realistic synthetic images, posing significant challenges to digital media authenticity. These generators are typically based on a few core architectural families, primarily Generative Adversarial Networks (GANs) and Diffusion Models (DMs). A critical vulnerability in current forensics is the failure of detectors to achieve cross-generator generalization, especially when crossing architectural boundaries (e.g., from GANs to DMs). We hypothesize that this gap stems from fundamental differences in the artifacts produced by these \textbf{distinct architectures}. In this work, we provide a theoretical analysis explaining how the distinct optimization objectives of the GAN and DM architectures lead to different manifold coverage behaviors. We demonstrate that GANs permit partial coverage, often leading to boundary artifacts, while DMs enforce complete coverage, resulting in over-smoothing patterns. Motivated by this analysis, we propose the \textbf{Tri}archy \textbf{Detect}or (TriDetect), a semi-supervised approach that enhances binary classification by discovering latent architectural patterns within the "fake" class. TriDetect employs balanced cluster assignment via the Sinkhorn-Knopp algorithm and a cross-view consistency mechanism, encouraging the model to learn fundamental architectural distincts. We evaluate our approach on two standard benchmarks and three in-the-wild datasets against 13 baselines to demonstrate its generalization capability to unseen generators.

Beyond Binary Classification: A Semi-supervised Approach to Generalized AI-generated Image Detection

TL;DR

This work addresses the challenge of detecting AI-generated images with limited cross-generator generalization, especially across GANs and diffusion models. It provides a theoretical analysis showing that GANs optimize the Jensen-Shannon divergence while diffusion models minimize a KL-divergence bound, leading to partial vs complete data-manifold coverage and distinct latent artifacts. Based on these insights, the authors propose TriDetect, a semi-supervised detector that jointly performs real/fake classification and latent-architecture pattern discovery using balanced optimal-transport clustering (via Sinkhorn-Knopp) and cross-view consistency. Extensive experiments across GenImage, AIGCDetectBenchmark, and in-the-wild datasets demonstrate TriDetect’s strong generalization to unseen generators within the same architectural families and robustness to degradations, highlighting a new architecture-centered paradigm for AI-generated image detection.

Abstract

The rapid advancement of generators (e.g., StyleGAN, Midjourney, DALL-E) has produced highly realistic synthetic images, posing significant challenges to digital media authenticity. These generators are typically based on a few core architectural families, primarily Generative Adversarial Networks (GANs) and Diffusion Models (DMs). A critical vulnerability in current forensics is the failure of detectors to achieve cross-generator generalization, especially when crossing architectural boundaries (e.g., from GANs to DMs). We hypothesize that this gap stems from fundamental differences in the artifacts produced by these \textbf{distinct architectures}. In this work, we provide a theoretical analysis explaining how the distinct optimization objectives of the GAN and DM architectures lead to different manifold coverage behaviors. We demonstrate that GANs permit partial coverage, often leading to boundary artifacts, while DMs enforce complete coverage, resulting in over-smoothing patterns. Motivated by this analysis, we propose the \textbf{Tri}archy \textbf{Detect}or (TriDetect), a semi-supervised approach that enhances binary classification by discovering latent architectural patterns within the "fake" class. TriDetect employs balanced cluster assignment via the Sinkhorn-Knopp algorithm and a cross-view consistency mechanism, encouraging the model to learn fundamental architectural distincts. We evaluate our approach on two standard benchmarks and three in-the-wild datasets against 13 baselines to demonstrate its generalization capability to unseen generators.

Paper Structure

This paper contains 39 sections, 3 theorems, 48 equations, 2 figures, 21 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. From Divergence to Detection: A Theoretical Analysis of Architectural Artifacts
  3. The Fundamental Disparity in Optimization
  4. From Divergence to Distribution Support and Artifacts
  5. Methodology
  6. Problem Formulation and Objectives
  7. Balanced Clustering via Optimal Transport
  8. Cross-view Consistency for Generalized Learning
  9. Loss Function
  10. Experiments
  11. Experimental Setup
  12. Experimental Results
  13. Standard Benchmarks.
  14. In-the-wild Datasets.
  15. Ablation Study
  16. ...and 24 more sections

Key Result

Lemma 1

(DM Optimization Objective). Suppose that $x_0 \sim p_{\text{data}}$, DM minimizes an upper bound on the KL divergence between the data and model distributions: where ELBO is the Evidence Lower Bound and $H(p_{\text{data}})$ is the entropy of the data distribution.

Figures (2)

  • Figure 1: Overview of our proposed method, TriDetect.
  • Figure 2: Visualization of learned representations demonstrating successful discovery of fake sub-types. The three t-SNE projections display feature embeddings colored by (left) the model's unsupervised cluster assignments, (middle) the model's binary real/fake predictions, and (right) the ground-truth generation methods. Results are performed on AIGCDetectBenchmark.

Theorems & Definitions (13)

  • Remark 1
  • Lemma 2
  • Theorem 1
  • Theorem 2
  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • proof
  • ...and 3 more