Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models

TL;DR

The paper analyzes training-free AI-generated image detection through the lens of vision foundation model robustness, revealing that self-supervised backbones like DINOv2 yield embeddings more amenable to anomaly detection under perturbations. It shows Gaussian blur can outperform Gaussian noise for facial images due to frequency artifacts, and introduces Contrastive Blur to amplify embedding separation, plus MINDER to balance cross-domain performance. The proposed framework achieves state-of-the-art training-free detection on facial and general image datasets, with ablations demonstrating additive gains from combining components. These findings offer practical guidance for deploying robust deepfake detectors and contribute to understanding how model robustness properties drive effective detection. The work highlights ongoing challenges, including local boundary artifacts and sensitivity to JPEG compression, pointing to future directions in multi-noise strategies and domain-balanced detection.

Abstract

The rapid advancement of generative models has introduced serious risks, including deepfake techniques for facial synthesis and editing. Traditional approaches rely on training classifiers and enhancing generalizability through various feature extraction techniques. Meanwhile, training-free detection methods address issues like limited data and overfitting by directly leveraging statistical properties from vision foundation models to distinguish between real and fake images. The current leading training-free approach, RIGID, utilizes DINOv2 sensitivity to perturbations in image space for detecting fake images, with fake image embeddings exhibiting greater sensitivity than those of real images. This observation prompts us to investigate how detection performance varies across model backbones, perturbation types, and datasets. Our experiments reveal that detection performance is closely linked to model robustness, with self-supervised (SSL) models providing more reliable representations. While Gaussian noise effectively detects general objects, it performs worse on facial images, whereas Gaussian blur is more effective due to potential frequency artifacts. To further improve detection, we introduce Contrastive Blur, which enhances performance on facial images, and MINDER (MINimum distance DetEctoR), which addresses noise type bias, balancing performance across domains. Beyond performance gains, our work offers valuable insights for both the generative and detection communities, contributing to a deeper understanding of model robustness property utilized for deepfake detection.

Figures (6)

• Figure 1: Grad-CAM gradcam of Feature Similarity with respect to Noise Across Different Perturbations. Grad-CAM highlights the regions of interest (ROIs) in the output embedding of DINOv2, which are responsive to various types of added noise (at the pixel level). Note that we visualize the average Grad-CAM on facial images from MidJourney-6 midjourney and CollabDiff collabdiff datasets, respectively.
• Figure 2: Frequency spectrum visualization of real and fake images. Images from CollabDiff collabdiff and ImageNet imagenet contain more high-frequency component, leading to greater perturbation. The notation $\mathcal{F}(\cdot)$ refers to the discrete Fourier transform and $X$ represents the image dataset. The frequency maps are center-shifted and smoothed using $\log{(1 + x)}$ after being averaged over the entire dataset.
• Figure 3: Framework Overview. Our framework extend the existing method RIGID rigid by incorporating Contrastive Blur (see Sec.\ref{['sec:contrastive']}) and MINDER (see Sec. \ref{['sec:minder']}). This enhancement improves deepfake image detection performance and mitigates detection biases that arise from applying different types of noise. The results, shown in the bar plot, indicate a significant improvement in the average AUROC across two domains. In the figure, $f(x)$ denotes the vision encoder DINOv2. The fake images (the second and third rows) are also challenging to distinguish with the human eye.
• Figure 4: Distribution Visualization of Cosine Similarity after Perturbations. Blurring produces a gap in the cosine similarity distribution for facial images generated by CollabDiff collabdiff, while for general images in ADM adm, the distribution becomes indistinguishable. This trend reverses when applying Gaussian noise.
• Figure 5: Performance degradation under JPEG compression. As expected, Gaussian blur is more sensitive to frequency corruption, leading to a larger decrease in performance. However, MINDER remains robust in effectively leveraging both types of noise, even though the individual methods it combines are affected by JPEG compression.