How Noise Benefits AI-generated Image Detection

TL;DR

This work tackles the open-world generalization problem in AI-generated image detection, where detectors overfit to shortcuts in training data. It introduces PiN-CLIP, a variational, noise-driven framework that jointly learns a noise generator and a CLIP-based detector, using cross-attention to produce artifact-conditioned perturbations that maximize $I(\mathcal{T};\mathcal{E})$ and reduce $H(\mathcal{T}|\mathcal{E})$. By injecting mean-guided and curvature-aware noise into the feature space, PiN-CLIP suppresses shortcut-prone directions while amplifying stable forensic cues, achieving state-of-the-art cross-domain performance on GenImage and AIGIBench, with notable improvements in average accuracy. The approach demonstrates strong robustness to common perturbations (JPEG, blur) and provides a principled, noise-driven paradigm for improving multimedia forensics in open-world settings.

Abstract

The rapid advancement of generative models has made real and synthetic images increasingly indistinguishable. Although extensive efforts have been devoted to detecting AI-generated images, out-of-distribution generalization remains a persistent challenge. We trace this weakness to spurious shortcuts exploited during training and we also observe that small feature-space perturbations can mitigate shortcut dominance. To address this problem in a more controllable manner, we propose the Positive-Incentive Noise for CLIP (PiN-CLIP), which jointly trains a noise generator and a detection network under a variational positive-incentive principle. Specifically, we construct positive-incentive noise in the feature space via cross-attention fusion of visual and categorical semantic features. During optimization, the noise is injected into the feature space to fine-tune the visual encoder, suppressing shortcut-sensitive directions while amplifying stable forensic cues, thereby enabling the extraction of more robust and generalized artifact representations. Comparative experiments are conducted on an open-world dataset comprising synthetic images generated by 42 distinct generative models. Our method achieves new state-of-the-art performance, with notable improvements of 5.4 in average accuracy over existing approaches.

Figures (3)

• Figure 1: The Cross-entropy Loss during Training. When training the CLIP-LoRA network, the loss rapidly drops to 0.1 within only about 100 iterations, indicating early overfitting. Introducing tiny feature-space perturbations (CLIP-Random) mitigates this effect to some extent, maintaining a higher loss of around 0.18 over the same iterations. In contrast, our proposed PiN-CLIP slows the early descent and stabilizes margins throughout training, consistent with suppressing shortcut-sensitive directions and promoting more robust convergence.
• Figure 2: Architecture of PiN-CLIP for Generalizable AI-Generated Image Detection. It includes a conditional noise generator and a detection network, enabling task-adaptive, forgery-conditional noise injection guided by the optimization objective. This mechanism suppresses shortcut-sensitive directions while amplifying stable, task-relevant forensic evidence under beneficial stochastic transformations.
• Figure 3: T-SNE Visualization of Features Extracted Using CLIP, CLIP-random and Ours. Our method achieves strong real/fake discrimination, while adding random noise to the visual features also improves discriminative ability.