Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing

TL;DR

FaceLock addresses privacy risks from diffusion-based portrait editing by softening biometric identity after edits rather than suppressing edits themselves. It introduces an in-loop perturbation optimization that jointly targets a facial-recognition loss and a feature-embedding disparity within the diffusion process, yielding robust protection across diverse prompts. The work also critiques widely used image-editing metrics and proposes perceptual and biometric integrity measures to better evaluate protection efficacy. Empirical results show FaceLock outperforms prior defenses and remains robust under common purification techniques, highlighting its practical potential for privacy-preserving editing workflows.

Abstract

Recent advancements in diffusion models have made generative image editing more accessible, enabling creative edits but raising ethical concerns, particularly regarding malicious edits to human portraits that threaten privacy and identity security. Existing protection methods primarily rely on adversarial perturbations to nullify edits but often fail against diverse editing requests. We propose FaceLock, a novel approach to portrait protection that optimizes adversarial perturbations to destroy or significantly alter biometric information, rendering edited outputs biometrically unrecognizable. FaceLock integrates facial recognition and visual perception into perturbation optimization to provide robust protection against various editing attempts. We also highlight flaws in commonly used evaluation metrics and reveal how they can be manipulated, emphasizing the need for reliable assessments of protection. Experiments show FaceLock outperforms baselines in defending against malicious edits and is robust against purification techniques. Ablation studies confirm its stability and broad applicability across diffusion-based editing algorithms. Our work advances biometric defense and sets the foundation for privacy-preserving practices in image editing. The code is available at: https://github.com/taco-group/FaceLock.

Figures (12)

• Figure 1: An illustration of adversarial perturbation generation for safeguarding personal images from malicious editing. Perturbations generated by prior work salman2023raisingchen2023editshield aim to cancel off editing effects, resulting in instability due to the diversity of editing instructions. In contrast, FaceLock does not prevent edits from being applied but instead erases critical biometric information (e.g., human facial features) after editing, making it agnostic to specific prompts and achieving superior performance.
• Figure 2: Illustration of the two requirements of image editing task: prompt fidelity and image integrity. (a) Source image before editing; (b) A successful editing example holding both metrics; (c) Failure case due to the lack of prompt fidelity leading to under-editing and (d) the lack of the image integrity leading to over-editing.
• Figure 3: Source and edited images generated from different protection methods based on the instruction “Let the person wear a police suit”. The FR score below each image represents the facial representation similarity between the edited and source images and scores marked in red indicate insignificant changes biometric recognition results by CVLFace compared to source image. 'CVL' refers to perturbations generated targeting the CVLFace model alone. 'CVL-D' represents protection targeting both the CVLFace model and the diffusion model, while 'CVL-DP' incorporates an auxiliary loss to enforce pixel-level disparity between the edited and source images. FaceLock targets the CVLFace and diffusion model, aiming to enhance the disparity between the feature embeddings of the decoded and original images.
• Figure 4: CLIP score (CLIP-S) of different editing results. The CLIP score provides a contradictory ranking (III > II > I) compared to the visual quality (I > II > III), as it overemphasizes the presence of elements from the editing prompt, thereby favoring over-editing.
• Figure 5: SSIM and PSNR scores of different defense methods. Although Defense I (b) demonstrates a successful defense, Defense II (d) is assigned a much lower (better) SSIM and PSNR score simply due to its larger pixel-level statistical difference from (b). SSIM and PSNR treat the edited image w/o defense as the gold standard, without accounting for the diversity of possible editing outcomes, which can lead to a false sense of defense success.