Privacy-preserving Optics for Enhancing Protection in Face De-identification

TL;DR

The paper tackles privacy in face de-identification by shifting from software-only approaches to a hardware-level solution that learns optics to hide identity while preserving geometric cues. It introduces a two-stage framework: Stage I learns a phase-mask-based optical encoder and a heatmap regressor to retain pose and landmark information, while Stage II employs a GAN conditioned on the heatmap and a public reference image to synthesize a new identity. Evaluations on FFHQ/CelebA-HQ datasets and a physical prototype demonstrate improved privacy protection against deconvolution attacks and competitive image quality, with human studies underscoring effective anonymization. Key contributions include end-to-end optics learning using a Zernike-based phase mask, a heatmap-guided geometry preservation approach, and a reference-guided, privacy-aware face synthesis pipeline that mitigates sniffing risks at capture time.

Abstract

The modern surge in camera usage alongside widespread computer vision technology applications poses significant privacy and security concerns. Current artificial intelligence (AI) technologies aid in recognizing relevant events and assisting in daily tasks in homes, offices, hospitals, etc. The need to access or process personal information for these purposes raises privacy concerns. While software-level solutions like face de-identification provide a good privacy/utility trade-off, they present vulnerabilities to sniffing attacks. In this paper, we propose a hardware-level face de-identification method to solve this vulnerability. Specifically, our approach first learns an optical encoder along with a regression model to obtain a face heatmap while hiding the face identity from the source image. We also propose an anonymization framework that generates a new face using the privacy-preserving image, face heatmap, and a reference face image from a public dataset as input. We validate our approach with extensive simulations and hardware experiments.

Figures (6)

• Figure 1: Traditional vs. our proposed pipeline. (a) The traditional face de-identification pipeline is vulnerable to adversarial attacks (unlock). (b) We propose to close the security gap and enhance protection by learning privacy-preserving optics (lock).
• Figure 2: Our proposed face de-identification framework. We first jointly optimize the camera optics and a heatmap regression network to obtain a face heatmap and conceal source image identity. Then, we train a GAN network to generate a new face identity using the acquired image, the obtained heatmap, and a reference image. We leverage global geometry information (e.g. face pose and eyes, nose, mouth position) from the acquired image and the heatmap while the style is extracted from the reference image.
• Figure 3: Qualitative results. We qualitatively evaluate our proposed face de-identification method using the CelebA-HQ dataset.
• Figure 4: Qualitative Results. We compare our method against SOTA methods: CIAGAN, FIT, DeepPrivacy, and RIDDLE.
• Figure 5: Robustness to Deconvolution. We use the SOTA diffusion model in kawar2022denoising to recover the person identity. We show the PSNR between the original and recovered face in each image.