Perturb, Attend, Detect and Localize (PADL): Robust Proactive Image Defense

TL;DR

Unlike previous proactive defenses that rely on a finite set of perturbations, PADL’s tailored protection significantly reduces the risk of reverse engineering, ensuring its effectiveness across diverse media formats and real-world applications.

Abstract

Image manipulation detection and localization have received considerable attention from the research community given the blooming of Generative Models (GMs). Detection methods that follow a passive approach may overfit to specific GMs, limiting their application in real-world scenarios, due to the growing diversity of generative models. Recently, approaches based on a proactive framework have shown the possibility of dealing with this limitation. However, these methods suffer from two main limitations, which raises concerns about potential vulnerabilities: i) the manipulation detector is not robust to noise and hence can be easily fooled; ii) the fact that they rely on fixed perturbations for image protection offers a predictable exploit for malicious attackers, enabling them to reverse-engineer and evade detection. To overcome this issue we propose PADL, a new solution able to generate image-specific perturbations using a symmetric scheme of encoding and decoding based on cross-attention, which drastically reduces the possibility of reverse engineering, even when evaluated with adaptive attack [31]. Additionally, PADL is able to pinpoint manipulated areas, facilitating the identification of specific regions that have undergone alterations, and has more generalization power than prior art on held-out generative models. Indeed, although being trained only on an attribute manipulation GAN model [15], our method generalizes to a range of unseen models with diverse architectural designs, such as StarGANv2, BlendGAN, DiffAE, StableDiffusion and StableDiffusionXL. Additionally, we introduce a novel evaluation protocol, which offers a fair evaluation of localisation performance in function of detection accuracy and better captures real-world scenarios.

Figures (10)

• Figure 1: Attack to proactive defense methods.left:(a) When simple Gaussian noise with increasing $\sigma$ is added to an image, the solution from asnani2023malp will detect these images as protected, whereas PADL demonstrates its robustness to noise. (b) by using the process on the right we were able to reverse engineer the perturbation of asnani2023malp and use it to protect new arbitrary images, achieving a high detection accuracy. (c) we apply the same attack to our solution which remains robust even when a larger collection of protected images is provided. right: Our attack uses a fixed set of $K$ protected images to reverse the protection of a proactive method. All results have been obtained by averaging over 10 trials.
• Figure 2: Architecture overview. The encoding module creates a specific perturbation and adds it to a real image for protection. The decoding module first estimates the perturbation and then uses it to perform manipulation detection and localization.
• Figure 3: Comparison of protected images. (a) original images (top), images protected by asnani2023malp (middle) and by PADL (bottom). Zoom in for better visualization. (b) Image quality measured in terms of MSE and LPIPS both calculated between real images and their protected version.
• Figure 4: Loss Ablation: (Left) The model is unable to generate different perturbations without $\mathcal{L}_{div}$. (Right) Without the max in $\mathcal{L}_{div}$ the model learns only two perturbations in opposite directions.
• Figure 5: Qualitative comparison of protected images with different protection strengths. Progression of the visual quality of protected images with an increasing value of $\alpha$. Values over $0.05$ result in visible artifacts which compromise the image quality. Zoom in for better visualization.