Fixed-Threshold Evaluation of a Hybrid CNN-ViT for AI-Generated Image Detection Across Photos and Art

TL;DR

This work addresses deployment realism in AI-generated image detection by introducing a fixed-threshold evaluation protocol that keeps decision boundaries constant across post-processing transformations. It presents a lightweight CNN–ViT hybrid with gated fusion and optional frequency enhancement, showing that CNNs excel on pristine photos but are fragile to compression, ViTs remain robust through semantic patterns, and hybrids balance cross-domain performance. Across three deployment operating points and multiple datasets, the study reveals a forensic–semantic spectrum with artistic content yielding higher AUROC than photorealistic content and reports strong CIFAKE performance (AUROC ≈ 0.9977). The key contributions are the fixed-threshold protocol, statistically validated cross-domain insights, and practical deployment guidance for choosing architectures by content type and degradation. The findings have immediate implications for real-world detectors in journalism, moderation, and digital provenance, guiding when to deploy CNNs, ViTs, or hybrids depending on content and post-processing conditions.

Abstract

AI image generators create both photorealistic images and stylized art, necessitating robust detectors that maintain performance under common post-processing transformations (JPEG compression, blur, downscaling). Existing methods optimize single metrics without addressing deployment-critical factors such as operating point selection and fixed-threshold robustness. This work addresses misleading robustness estimates by introducing a fixed-threshold evaluation protocol that holds decision thresholds, selected once on clean validation data, fixed across all post-processing transformations. Traditional methods retune thresholds per condition, artificially inflating robustness estimates and masking deployment failures. We report deployment-relevant performance at three operating points (Low-FPR, ROC-optimal, Best-F1) under systematic degradation testing using a lightweight CNN-ViT hybrid with gated fusion and optional frequency enhancement. Our evaluation exposes a statistically validated forensic-semantic spectrum: frequency-aided CNNs excel on pristine photos but collapse under compression (93.33% to 61.49%), whereas ViTs degrade minimally (92.86% to 88.36%) through robust semantic pattern recognition. Multi-seed experiments demonstrate that all architectures achieve 15% higher AUROC on artistic content (0.901-0.907) versus photorealistic images (0.747-0.759), confirming that semantic patterns provide fundamentally more reliable detection cues than forensic artifacts. Our hybrid approach achieves balanced cross-domain performance: 91.4% accuracy on tiny-genimage photos, 89.7% on AiArtData art/graphics, and 98.3% (competitive) on CIFAKE. Fixed-threshold evaluation eliminates retuning inflation, reveals genuine robustness gaps, and yields actionable deployment guidance: prefer CNNs for clean photo verification, ViTs for compressed content, and hybrids for art/graphics screening.

Figures (5)

• Figure 1: Hybrid CNN–ViT architecture with adaptive gating. Dual-branch processing extracts local forensic features via ResNet-50 (2048-D) and global semantic patterns via ViT-B/16 (768-D), with optional frequency enhancement (FFT). Projected features ($z_c$, $z_v$) are adaptively weighted ($w_c$, $w_v$) and fused as $\tilde{z} = w_c \cdot z_c + w_v \cdot z_v$ for content-adaptive detection.
• Figure 2: Operating point performance. (a) On photos (tiny-genimage), CNNs excel at strict thresholds. (b) On art/graphics (AiArtData/RealArt), the Hybrid architecture consistently outperforms baselines.
• Figure 3: Cross-domain performance comparison. (a) AUROC scores showing higher discriminability on art vs. photos. (b) Accuracy at ROC-optimal thresholds, highlighting CNN superiority on photos(0.933) and Hybrid dominance on art/graphics (0.897).
• Figure 4: Fixed-threshold robustness analysis on photos. CNN accuracy collapses from 93.3% (clean) to 61.5% (JPEG Q60) as forensic artifacts degrade, while ViT maintains 88.4% through semantic patterns. Blur causes severe CNN degradation (56.7% at $\sigma$=3) versus modest ViT decline (78.8%).
• Figure 5: CIFAKE benchmark comparison. (a) Accuracy and (b) AUROC analysis showing our Hybrid architecture achieves 98.32% accuracy and 0.9977 AUROC, matching state-of-the-art DenseNet-121 performance.