Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Skin Color Driven Privacy Preserving Face Recognition via Function Secret Sharing

Dong Han, Yufan Jiang, Yong Li, Ricardo Mendes, Joachim Denzler

TL;DR

The paper tackles privacy-preserving face recognition by augmenting encrypted embeddings with pure skin color patches, forming a fused representation that improves accuracy. It introduces a Function Secret Sharing (FSS)–based embedding distance protocol to securely compare protected embeddings against a threshold, avoiding leakage of intermediate results while boosting online efficiency. Empirical results show competitive 1:1 accuracy and superior 1:N retrieval, with robustness to black-box attacks and substantial speedups over SS-based approaches in both LAN and WAN settings. The work also discusses security limitations and practical mitigations, framing a path toward integration with secure hardware and lighter feature extractors for broader deployment. $c = \frac{P_c \cdot P_i}{\|P_c\|\|P_i\|}$ and $\mathsf{th}$ are central to the comparison, and the approach leverages this with masked shares and $\Pi^\mathsf{FSS}$ gates to preserve privacy during distance computation.

Abstract

In this work, we leverage the pure skin color patch from the face image as the additional information to train an auxiliary skin color feature extractor and face recognition model in parallel to improve performance of state-of-the-art (SOTA) privacy-preserving face recognition (PPFR) systems. Our solution is robust against black-box attacking and well-established generative adversarial network (GAN) based image restoration. We analyze the potential risk in previous work, where the proposed cosine similarity computation might directly leak the protected precomputed embedding stored on the server side. We propose a Function Secret Sharing (FSS) based face embedding comparison protocol without any intermediate result leakage. In addition, we show in experiments that the proposed protocol is more efficient compared to the Secret Sharing (SS) based protocol.

Robust Skin Color Driven Privacy Preserving Face Recognition via Function Secret Sharing

TL;DR

The paper tackles privacy-preserving face recognition by augmenting encrypted embeddings with pure skin color patches, forming a fused representation that improves accuracy. It introduces a Function Secret Sharing (FSS)–based embedding distance protocol to securely compare protected embeddings against a threshold, avoiding leakage of intermediate results while boosting online efficiency. Empirical results show competitive 1:1 accuracy and superior 1:N retrieval, with robustness to black-box attacks and substantial speedups over SS-based approaches in both LAN and WAN settings. The work also discusses security limitations and practical mitigations, framing a path toward integration with secure hardware and lighter feature extractors for broader deployment. and are central to the comparison, and the approach leverages this with masked shares and gates to preserve privacy during distance computation.

Abstract

In this work, we leverage the pure skin color patch from the face image as the additional information to train an auxiliary skin color feature extractor and face recognition model in parallel to improve performance of state-of-the-art (SOTA) privacy-preserving face recognition (PPFR) systems. Our solution is robust against black-box attacking and well-established generative adversarial network (GAN) based image restoration. We analyze the potential risk in previous work, where the proposed cosine similarity computation might directly leak the protected precomputed embedding stored on the server side. We propose a Function Secret Sharing (FSS) based face embedding comparison protocol without any intermediate result leakage. In addition, we show in experiments that the proposed protocol is more efficient compared to the Secret Sharing (SS) based protocol.
Paper Structure (12 sections, 1 equation, 5 figures, 3 tables)

This paper contains 12 sections, 1 equation, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. PROPOSED METHOD
  3. Skin Color Information for PPFR
  4. FFS-based Face Embedding Distance Comparison
  5. How the Security in HFCF-DLBP Fails
  6. An FSS-based Solution
  7. EXPERIMENTS
  8. Datasets and Implementation Details
  9. Performance Comparisons
  10. Black-box Attacking
  11. Benchmark FSS
  12. Conclusion

Figures (5)

  • Figure 1: Overview of proposed HFCF-SkinColor privacy-preserving FR framework with FSS-based Distance Computation.
  • Figure 2: Face embedding distance comparison protocol $\Pi^\mathsf{FSS}_\mathsf{EmComp}$ via FSS
  • Figure 3: Face reconstruction by black-box attacking. From top to bottom, the target model are DCTDP, DLBPDP, HFCF-DLBP and Ours. The last row are ground truth images shown in gray color for better visualization.
  • Figure 4: Test image and DLBP representation.
  • Figure 5: Image restoration on a UNet reconstructed image. From top to bottom, target models are DCTDP, DLBPDP, HFCF-DLBP and Ours. The test image, results from UNet, results after applying the wiener filter and GFP-GAN are shown from left to right. The most right column contains the results from GFP-GAN when UNet reconstruct image without adding DP noise.