Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Privacy in Face Analytics Using Fully Homomorphic Encryption

Bharat Yalavarthi, Arjun Ramesh Kaushik, Arun Ross, Vishnu Boddeti, Nalini Ratha

TL;DR

The paper addresses the risk of soft biometrics leakage from face embeddings by proposing a privacy-preserving framework that combines Fully Homomorphic Encryption (FHE) with PolyProtect template protection and embedding compression. The pipeline encrypts embeddings, applies a polynomial-based protection, and operates entirely in the encrypted domain, preserving identification performance while blocking leakage of attributes such as age, gender, and ethnicity. It introduces Matryoshka Representation Learning (MRL) for embedding compression, analyzes ablation studies over PolyProtect parameters, and demonstrates robustness across datasets (CelebSet and BFW) and embedding backbones (FaceNet and AdaFace). The approach offers irreversibility and unlinkability under full disclosure threat models, showing practical potential for secure face analytics in privacy-constrained settings.

Abstract

Modern face recognition systems utilize deep neural networks to extract salient features from a face. These features denote embeddings in latent space and are often stored as templates in a face recognition system. These embeddings are susceptible to data leakage and, in some cases, can even be used to reconstruct the original face image. To prevent compromising identities, template protection schemes are commonly employed. However, these schemes may still not prevent the leakage of soft biometric information such as age, gender and race. To alleviate this issue, we propose a novel technique that combines Fully Homomorphic Encryption (FHE) with an existing template protection scheme known as PolyProtect. We show that the embeddings can be compressed and encrypted using FHE and transformed into a secure PolyProtect template using polynomial transformation, for additional protection. We demonstrate the efficacy of the proposed approach through extensive experiments on multiple datasets. Our proposed approach ensures irreversibility and unlinkability, effectively preventing the leakage of soft biometric attributes from face embeddings without compromising recognition accuracy.

Enhancing Privacy in Face Analytics Using Fully Homomorphic Encryption

TL;DR

The paper addresses the risk of soft biometrics leakage from face embeddings by proposing a privacy-preserving framework that combines Fully Homomorphic Encryption (FHE) with PolyProtect template protection and embedding compression. The pipeline encrypts embeddings, applies a polynomial-based protection, and operates entirely in the encrypted domain, preserving identification performance while blocking leakage of attributes such as age, gender, and ethnicity. It introduces Matryoshka Representation Learning (MRL) for embedding compression, analyzes ablation studies over PolyProtect parameters, and demonstrates robustness across datasets (CelebSet and BFW) and embedding backbones (FaceNet and AdaFace). The approach offers irreversibility and unlinkability under full disclosure threat models, showing practical potential for secure face analytics in privacy-constrained settings.

Abstract

Modern face recognition systems utilize deep neural networks to extract salient features from a face. These features denote embeddings in latent space and are often stored as templates in a face recognition system. These embeddings are susceptible to data leakage and, in some cases, can even be used to reconstruct the original face image. To prevent compromising identities, template protection schemes are commonly employed. However, these schemes may still not prevent the leakage of soft biometric information such as age, gender and race. To alleviate this issue, we propose a novel technique that combines Fully Homomorphic Encryption (FHE) with an existing template protection scheme known as PolyProtect. We show that the embeddings can be compressed and encrypted using FHE and transformed into a secure PolyProtect template using polynomial transformation, for additional protection. We demonstrate the efficacy of the proposed approach through extensive experiments on multiple datasets. Our proposed approach ensures irreversibility and unlinkability, effectively preventing the leakage of soft biometric attributes from face embeddings without compromising recognition accuracy.
Paper Structure (19 sections, 4 equations, 8 figures, 5 tables, 3 algorithms)

This paper contains 19 sections, 4 equations, 8 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Threat Model
  3. FHE Basics
  4. Related Work
  5. Template Protection
  6. Ablation Study
  7. Variation of user-defined parameters ($C$, $m$, $overlap$)
  8. Summation of Ciphertext elements
  9. Naive Rotation
  10. Discrete Fourier Transform
  11. Fold and Add
  12. Polynomial approximation of inverse square root
  13. Proposed Approach
  14. Dataset
  15. Embedding Compression
  16. ...and 4 more sections

Figures (8)

  • Figure 1: An overview of our contribution in protecting face embeddings from leaking soft biometric attributes - Age, Gender, Ethnicity (FHE - Fully Homomorphic Encryption).
  • Figure 2: Ablation study with the PolyProtect parameters shows soft biometrics leakage in different settings. (a) Overlap. (b) m - length of polynomial coefficients/exponents. (c) [-C, C] - range of polynomial coefficients.
  • Figure 3: Execution time to compute summation of elements within a ciphertext.
  • Figure 4: Inverse square root function. (a) 6-degree polynomial approximation and (b) 8-degree polynomial approximation and their relative error over 2000 random points in the range (0,1].
  • Figure 5: Performance of Matryoksha Representation Learning (MRL) in extracting features - Identity, Age, Gender, Ethnicity - from different compressed dimensions. The graph is based on experiments performed using AdaFace on the CelebSet dataset.
  • ...and 3 more figures