Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks

TL;DR

This work analyzes the robustness of AI-image detectors, focusing on watermark-based and classifier-based approaches. It introduces diffusion purification as a certified attack against low-perturbation watermarks and develops a model-substitution adversarial attack for high-perturbation watermarks, alongside spoofing methods that degrade detector performance. The authors establish a fundamental robustness-reliability trade-off for deepfake detectors and validate their theory with extensive experiments on ImageNet-watermarked samples and the FaceForensics++ dataset. The findings illuminate practical limits of watermarking and guide the design of more robust AI-generated-content detectors while underscoring ethical considerations surrounding detection technologies.

Abstract

In light of recent advancements in generative AI models, it has become essential to distinguish genuine content from AI-generated one to prevent the malicious usage of fake materials as authentic ones and vice versa. Various techniques have been introduced for identifying AI-generated images, with watermarking emerging as a promising approach. In this paper, we analyze the robustness of various AI-image detectors including watermarking and classifier-based deepfake detectors. For watermarking methods that introduce subtle image perturbations (i.e., low perturbation budget methods), we reveal a fundamental trade-off between the evasion error rate (i.e., the fraction of watermarked images detected as non-watermarked ones) and the spoofing error rate (i.e., the fraction of non-watermarked images detected as watermarked ones) upon an application of diffusion purification attack. To validate our theoretical findings, we also provide empirical evidence demonstrating that diffusion purification effectively removes low perturbation budget watermarks by applying minimal changes to images. The diffusion purification attack is ineffective for high perturbation watermarking methods where notable changes are applied to images. In this case, we develop a model substitution adversarial attack that can successfully remove watermarks. Moreover, we show that watermarking methods are vulnerable to spoofing attacks where the attacker aims to have real images identified as watermarked ones, damaging the reputation of the developers. In particular, with black-box access to the watermarking method, a watermarked noise image can be generated and added to real images, causing them to be incorrectly classified as watermarked. Finally, we extend our theory to characterize a fundamental trade-off between the robustness and reliability of classifier-based deep fake detectors and demonstrate it through experiments.

Figures (19)

• Figure 1: Illustration of our attacks against image watermarking methods. Upper panel demonstrates the diffusion purification attack for low perturbation budget (imperceptible) watermarks. It adds Gaussian noise to images, creating an indistinguishable region, which results in a certified lower bound on the error of watermark detectors. Noisy images are then denoised using diffusion models. See Section \ref{['subsection:diffpure_theory']} for the definition of the used terms (e.g., $\mathcal{R}$, $\mathcal{F}$). Lower panel depicts our model substitute adversarial attack against high-perturbation budget watermarks. Our attack involves training a substitute classifier, conducting a PGD attack on the substitute model, and using these manipulated images to deceive the black-box watermark detector.
• Figure 2: Lower bound on the sum of evasion and spoofing errors of image watermarks against diffusion purification attack from Theorem \ref{['theorem:diffpure_theory']}. The beta schedule for the diffusion model is linear in the range $[0.0008, 0.0120]$.
• Figure 3: ROC curves for empirical robustness of image watermark methods against diffusion purification attack with $t=0.2$. The dashed lines show the ROC curves of methods without attacking them.
• Figure 4: AUROC of watermarking methods against diffusion purification attack for a range of $t$ values. As expected, the robustness of methods against this attack has a correlation with the average image $\ell_2$ distance from Table \ref{['tab:watermark_l2_dist']}.
• Figure 5: AUROC of high-perturbation watermarking methods against $\ell_\infty$ adversarial attack w.r.t adversarial perturbation size $\epsilon$. The colored dashed lines measure robustness against uniform random noise in the range $[-2\epsilon, 2\epsilon]$.

Theorems & Definitions (13)

• Definition 1: Evasion and Spoofing Errors
• Definition 2
• Theorem 1
• Definition 3: Robust Detector
• Theorem 2
• proof
• Lemma 1
• proof
• proof
• Lemma 2