Forbes: Face Obfuscation Rendering via Backpropagation Refinement Scheme

TL;DR

Forbes tackles privacy protection in facial datasets by producing obfuscated images that humans cannot recognize while machines can still extract identity and attributes. It introduces eight parameterized obfuscating transformations organized into a composite pipeline and optimizes their parameters at inference time via a backpropagation refinement scheme (BRS), minimizing the energy $\mathcal{E}=\mathcal{E}_U+\mathcal{E}_C+\mathcal{E}_D+\mathcal{E}_S$ to balance HI and MD. The method yields two implementations, Forbes-G and Forbes-T, for generic and task-specific MD, respectively, and demonstrates strong HI/MD performance across face verification, age estimation, gender classification, and attribute classification on public benchmarks, corroborated by a dedicated user study. Importantly, Forbes operates without training, relying on a fixed inference-time optimization that preserves machine-readable information while suppressing human recognizability, offering practical privacy safeguards for large-scale facial data usage. The work provides concrete ablations, analysis of block sizes and energy terms, and qualitative results indicating robustness across datasets, underscoring the potential of energy-based, backpropagation-guided obfuscation in real-world privacy-preserving vision pipelines.

Abstract

A novel algorithm for face obfuscation, called Forbes, which aims to obfuscate facial appearance recognizable by humans but preserve the identity and attributes decipherable by machines, is proposed in this paper. Forbes first applies multiple obfuscating transformations with random parameters to an image to remove the identity information distinguishable by humans. Then, it optimizes the parameters to make the transformed image decipherable by machines based on the backpropagation refinement scheme. Finally, it renders an obfuscated image by applying the transformations with the optimized parameters. Experimental results on various datasets demonstrate that Forbes achieves both human indecipherability and machine decipherability excellently. The source codes are available at https://github.com/mcljtkim/Forbes.

Figures (17)

• Figure 1: Top, facial images are selected from the Vimeo-90K xue2019toflow, DAVIS 2016 Perazzi2016davis, LFW huang2008lfw, CelebA liu2015celeba datasets. Bottom, the facial regions are obfuscated by the proposed Forbes algorithm.
• Figure 2: Illustration of differences between the existing attribute-preserving face anonymization algorithm barattin2023attribute, the existing face obfuscation algorithm yuan2022pro, and the proposed Forbes algorithm.
• Figure 3: An overview of the proposed Forbes algorithm. Each obfuscating transformation $f_n$ has a set $\Theta_n$ of parameters. Also, $\Phi_1$ and $\Phi_2$ are composed of superposing parameters. A randomly initialized set $\Theta = \Theta_1 \cup \cdots \cup \Theta_8 \cup \Phi_{\rm 1} \cup \Phi_{\rm 2}$ is optimized to $\Theta^*$ in the backward pass to balance HI and MD. To this end, energy functions $\mathcal{E}_U$, $\mathcal{E}_C$, $\mathcal{E}_D$, and $\mathcal{E}_S$ are minimized.
• Figure 4: Examples of transformed images by $f_n$.
• Figure 5: Comparison of obfuscated images with or without the BRS optimization.