Personalized Privacy Protection Mask Against Unauthorized Facial Recognition

TL;DR

This work tackles the privacy risk posed by unauthorized facial recognition on web-shared images by introducing Chameleon, a user-centric system that learns a per-user P3-Mask to protect all images of that person. The mask is generated offline through a combination of cross-image protection, perceptibility optimization, and focal-diversity ensemble optimization, enabling instant on-device protection with minimal impact on visual quality. A de-obfuscation key mechanism allows authorized FR services to restore recognition when consent or authorization is provided, supporting cost-effective and selective FR usage. Empirical results on two benchmarks show that Chameleon outperforms state-of-the-art anti-FR methods against unknown models, while remaining resilient to adaptive adversaries and maintaining high image quality and low computational cost. Overall, Chameleon provides practical, scalable privacy protection with robust cross-model performance and real-time deployment potential on edge devices.

Abstract

Face recognition (FR) can be abused for privacy intrusion. Governments, private companies, or even individual attackers can collect facial images by web scraping to build an FR system identifying human faces without their consent. This paper introduces Chameleon, which learns to generate a user-centric personalized privacy protection mask, coined as P3-Mask, to protect facial images against unauthorized FR with three salient features. First, we use a cross-image optimization to generate one P3-Mask for each user instead of tailoring facial perturbation for each facial image of a user. It enables efficient and instant protection even for users with limited computing resources. Second, we incorporate a perceptibility optimization to preserve the visual quality of the protected facial images. Third, we strengthen the robustness of P3-Mask against unknown FR models by integrating focal diversity-optimized ensemble learning into the mask generation process. Extensive experiments on two benchmark datasets show that Chameleon outperforms three state-of-the-art methods with instant protection and minimal degradation of image quality. Furthermore, Chameleon enables cost-effective FR authorization using the P3-Mask as a personalized de-obfuscation key, and it demonstrates high resilience against adaptive adversaries.

Figures (9)

• Figure 1: (Top) Without protection, the privacy intruder can build a face database by web scraping and identify the unknown face. (Bottom) Chameleon learns the facial signature of the user (protectee) to generate a P3-Mask, which can be applied to protect any facial images before sharing them online against unauthorized FR.
• Figure 2: An overview of Chameleon's two-phase workflow for offline learning to generate P3-Mask and online protection with P3-Mask for personalized facial signature masking. The P3-Mask can protect any facial images of the same person without further learning.
• Figure 3: The iterative generation of P3-Mask for a user. Chameleon goes through multiple facial images of the same user to optimize a P3-Mask with ML pipeline awareness.
• Figure 4: Focal diversity provides a strong indicator of protection effectiveness for selecting an ensemble.
• Figure 5: Out of $28$ options of two-model ensembles from a pool of eight models, the ensemble selected by our approach (green) leads to much better protection than a randomly selected one (red).