Privacy-Preserving Face Recognition Using Trainable Feature Subtraction

TL;DR

This work addresses privacy concerns in face recognition by introducing MinusFace, a trainable feature-subtraction framework that creates a visually uninformative yet identity-preserving representation. It generates a high-dimensional residue $r = x - x'$ from a regenerated image $X' = g(X)$ and leverages a DCT-based encoding $e$ with invertible decoding $d$, followed by random channel shuffling to produce $X_p$, the protection target. The approach optimizes a dual objective that preserves recognizability via a FR model on $r$ while minimizing visual information through the regeneration-based subtraction, yielding strong privacy against recovery attacks and competitive recognition accuracy across standard benchmarks. Extensive experiments show MinusFace outperforms competing transform-based PPFR methods in privacy protection (low recoverability) while maintaining accuracy close to unprotected baselines, with favorable efficiency and compatibility. The method's practicality is underscored by its public code release and its applicability to diverse FR backbones and losses.

Abstract

The widespread adoption of face recognition has led to increasing privacy concerns, as unauthorized access to face images can expose sensitive personal information. This paper explores face image protection against viewing and recovery attacks. Inspired by image compression, we propose creating a visually uninformative face image through feature subtraction between an original face and its model-produced regeneration. Recognizable identity features within the image are encouraged by co-training a recognition model on its high-dimensional feature representation. To enhance privacy, the high-dimensional representation is crafted through random channel shuffling, resulting in randomized recognizable images devoid of attacker-leverageable texture details. We distill our methodologies into a novel privacy-preserving face recognition method, MinusFace. Experiments demonstrate its high recognition accuracy and effective privacy protection. Its code is available at https://github.com/Tencent/TFace.

Figures (12)

• Figure 1: Comparison between SOTAs and MinusFace. (a) SOTAs gradually remove the most visually informative features. Inadequacy of removal can result in successful recovery, which undermines privacy. (b) MinusFace first obtains a fully visually uninformative residue representation, then improves its recognizability. It exhibits better privacy protection than all SOTAs.
• Figure 2: Examples of image compression. Subtle details like texture are removed from (a) the original image to obtain (c) the compressed ones. The removed (b) residual representations are visually uninformative, yet carry descriptive features of the origin.
• Figure 3: The core idea of MinusFace. Imitating image compression, a visually uninformative residue $R$ is generated from feature subtraction: the original face minus its regeneration. $R$ is also optimized with an FR model to preserve identity features.
• Figure 4: The MinusFace pipeline. (a) It centers around the idea of feature subtraction, where the protective representation$X_p$ is derived from the residue between the original face $X$ and its regeneration $X'$. Both regeneration and feature subtraction occur in high-dimension to preserve identity features within the trained residue $r$. (b) The residue $r$ further undergoes random channel shuffling and decoding to produce the protective representation $X_p$. (c-d) All face figures are experimentally obtained and illustrate their representations faithfully.
• Figure 5: By randomly shuffling (b) channels of $r$, $192!$ distinct (c) $X_p$ can be generated from (a) the same $X$. We exhibit some channels and $X_p$. Different $X_p$ possess random texture patterns that obfuscate the recovery, by the nature of channel shuffling.