SlerpFace: Face Template Protection via Spherical Linear Interpolation

TL;DR

This work addresses the risk that diffusion-model inversion can reveal identity and appearance from protected face templates. It introduces SlerpFace, a transform-based FTP that rotates templates toward a noise-like distribution on the feature hypersphere using spherical linear interpolation, with a learnable, grouped dropout mechanism to enhance irreversibility. The approach yields strong protection against inversion and other privacy attacks while preserving recognition accuracy and reducing computational cost relative to prior arts. The findings suggest practical improvements for privacy-preserving FR deployments, enabling revocability and unlinkability without sacrificing performance.

Abstract

Contemporary face recognition systems use feature templates extracted from face images to identify persons. To enhance privacy, face template protection techniques are widely employed to conceal sensitive identity and appearance information stored in the template. This paper identifies an emerging privacy attack form utilizing diffusion models that could nullify prior protection. The attack can synthesize high-quality, identity-preserving face images from templates, revealing persons' appearance. Based on studies of the diffusion model's generative capability, this paper proposes a defense by rotating templates to a noise-like distribution. This is achieved efficiently by spherically and linearly interpolating templates on their located hypersphere. This paper further proposes to group-wisely divide and drop out templates' feature dimensions, to enhance the irreversibility of rotated templates. The proposed techniques are concretized as a novel face template protection technique, SlerpFace. Extensive experiments show that SlerpFace provides satisfactory recognition accuracy and comprehensive protection against inversion and other attack forms, superior to prior arts.

Figures (6)

• Figure 1: Paradigm of DM inversion attacks. It receives templates as conditional contexts and synthesizes identity-descriptive images. While unprotected templates and prior FTP arts are experimentally found vulnerable to inversion attacks, this paper presents a novel SlerpFace method as an effective defense. It deteriorates DM to let it generate images with obfuscated facial semantics and lower similarity scores, hence preserving privacy.
• Figure 2: (a) DM's generative capability: Consistent identity faces from authentic templates, but deteriorates with noise templates, causing semantic variation. (b) Slerp rotation at $d$=3: Query, positive, and negative templates rotate in the same direction, maintaining margin differences.
• Figure 3: Pipeline of SlerpFace: (1) Train an FR model to extract and group feature maps as templates. (2) Protect templates by independently rotating feature groups toward a key template and applying random dropout based on learnable weights. (3) During inference, extract a query template. (4) Match it with enrolled templates using the same rotation and dropout.
• Figure 4: (a) Ranges of $\Delta_{\theta}$ for templates with different feature dimensions $d$. $\Delta_{\theta}$ gradually increases as $d$ decreases. (b) Random dropout. Each feature group randomly discards equal dimensions of features. (c) Weighted dropout. Feature groups with larger weights discard fewer feature dimensions to better preserve crucial features.
• Figure 5: Paradigm of learnable feature grouping: Modify the FR model's output layer to generate feature maps, which are split into groups. Self-attention layers provide group-wise weights. For face image pairs, $\mathcal{L}_{g}$ aligns group-wise weighted similarities with original template similarities, trained alongside $\mathcal{L}_{fr}$. During inference, feature maps are reorganized as templates.