Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Crafter: Facial Feature Crafting against Inversion-based Identity Theft on Deep Models

Shiming Wang, Zhe Ji, Liyao Xiang, Hao Zhang, Xinbing Wang, Chenghu Zhou, Bo Li

TL;DR

Crafter tackles identity leakage from edge-extracted facial features by crafting features at the edge to induce inversion results that resemble non-private priors. It introduces identity perceptual privacy and the $\epsilon$-PII metric, and solves a nested minimax objective via an implicit-function approach to keep utility while guiding attacker reconstructions toward average faces. The method demonstrates robustness against white-box, black-box, hybrid, and adaptive inversion attacks across CelebA, LFW, and VGGFace2, outperforming state-of-the-art adversarial game-based defenses. The approach is deployment-friendly (no backend changes) and open-sourced, offering a practical path to secure edge-cloud facial processing.

Abstract

With the increased capabilities at the edge (e.g., mobile device) and more stringent privacy requirement, it becomes a recent trend for deep learning-enabled applications to pre-process sensitive raw data at the edge and transmit the features to the backend cloud for further processing. A typical application is to run machine learning (ML) services on facial images collected from different individuals. To prevent identity theft, conventional methods commonly rely on an adversarial game-based approach to shed the identity information from the feature. However, such methods can not defend against adaptive attacks, in which an attacker takes a countermove against a known defence strategy. We propose Crafter, a feature crafting mechanism deployed at the edge, to protect the identity information from adaptive model inversion attacks while ensuring the ML tasks are properly carried out in the cloud. The key defence strategy is to mislead the attacker to a non-private prior from which the attacker gains little about the private identity. In this case, the crafted features act like poison training samples for attackers with adaptive model updates. Experimental results indicate that Crafter successfully defends both basic and possible adaptive attacks, which can not be achieved by state-of-the-art adversarial game-based methods.

Crafter: Facial Feature Crafting against Inversion-based Identity Theft on Deep Models

TL;DR

Crafter tackles identity leakage from edge-extracted facial features by crafting features at the edge to induce inversion results that resemble non-private priors. It introduces identity perceptual privacy and the -PII metric, and solves a nested minimax objective via an implicit-function approach to keep utility while guiding attacker reconstructions toward average faces. The method demonstrates robustness against white-box, black-box, hybrid, and adaptive inversion attacks across CelebA, LFW, and VGGFace2, outperforming state-of-the-art adversarial game-based defenses. The approach is deployment-friendly (no backend changes) and open-sourced, offering a practical path to secure edge-cloud facial processing.

Abstract

With the increased capabilities at the edge (e.g., mobile device) and more stringent privacy requirement, it becomes a recent trend for deep learning-enabled applications to pre-process sensitive raw data at the edge and transmit the features to the backend cloud for further processing. A typical application is to run machine learning (ML) services on facial images collected from different individuals. To prevent identity theft, conventional methods commonly rely on an adversarial game-based approach to shed the identity information from the feature. However, such methods can not defend against adaptive attacks, in which an attacker takes a countermove against a known defence strategy. We propose Crafter, a feature crafting mechanism deployed at the edge, to protect the identity information from adaptive model inversion attacks while ensuring the ML tasks are properly carried out in the cloud. The key defence strategy is to mislead the attacker to a non-private prior from which the attacker gains little about the private identity. In this case, the crafted features act like poison training samples for attackers with adaptive model updates. Experimental results indicate that Crafter successfully defends both basic and possible adaptive attacks, which can not be achieved by state-of-the-art adversarial game-based methods.
Paper Structure (46 sections, 3 theorems, 38 equations, 29 figures, 15 tables, 3 algorithms)

This paper contains 46 sections, 3 theorems, 38 equations, 29 figures, 15 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Setting and Threat Model
  4. Problem Setting
  5. Threat Model
  6. Methodology
  7. Design Choices
  8. Privacy-Preserving Feature Crafting
  9. Algorithm of Crafter
  10. $\epsilon$-Perceptual Inversion Indistinguishability
  11. Discussion
  12. Robustness against Adaptive Attacks
  13. Implicit Optimization
  14. Evaluation
  15. Setup
  16. ...and 31 more sections

Key Result

Lemma 1

For a function $f(x, y): \mathbb{R}^{n+m} \rightarrow \mathbb{R}^m$, if some $(a,b)$ satisfies then surrounding $(a, b)$ there exist $U \subset \mathbb{R}^n$ and a unique continuously differentiable function $g:U \rightarrow \mathbb{R}^m$ that $g(a) = b$ and $f(x, g(x))=0, \forall x\in U$. In addition, $\frac{\partial g}{x}(x)=-\left[J_{f, \mathbf{y}}(x, g(x))\right]^{-1}\left[\frac{\partial

Figures (29)

  • Figure 1: Conventional methods adopt a stay-away approach where the defence strategy is easily overwhelmed by an adaptive attacker step; our Crafter takes a get-close approach where the crafted features act like poison training samples to the adversary, disrupting the training of adaptive attackers.
  • Figure 2: Inversion results of existing defences and Crafter against multiple attacks. Crafter demonstrates robustness against both basic and possible adaptive attacks, while the baselines are not adaptive attacker-proof.
  • Figure 3: Problem overview. Users (blue) release locally encoded feature $Enc(X)$ of private image $X$ to complete computation tasks (black). Attackers intercept the released feature and attempt to reconstruct original private input through either black-box attack (brown) or white-box attack (red).
  • Figure 4: Overview of our feature crafting scheme against inversion attack. Attacker (red) obtains a best-response latent vector $z^*$ of protected feature $F_X$ by minimizing inversion loss $\mathcal{L}_{\mathrm{inv}}$. Defender (blue) manipulates $F_X$ to balance privacy $\mathcal{L}_{\mathrm{p}}$ of reconstructed $z^*$ and utility $\mathcal{L}_{\mathrm{u}}$ for computation tasks.
  • Figure 5: Visualization of the best-response $z_{\mathrm{org}}$ of the raw feature (red), $z^{*}$ of our crafted feature (blue), and $z_r$ the attacker's prior (green). Our framework shifts the unprotected posterior belief towards attacker's prior belief.
  • ...and 24 more figures

Theorems & Definitions (11)

  • Definition 1: KR duality of the Earth Mover's distance
  • Lemma 1: Cauchy, Implicit Function Theorem
  • Lemma 2: Lorraine lorraine2020optimizing, Neumann Inverse Approximation
  • Definition 2: $\epsilon$-Perceptual Inversion Indistinguishability
  • Definition 3: Adjacent datasets
  • Definition 4: $\epsilon$-Differential Privacy
  • Definition 5: Equivalence class
  • Definition 6: The $t$-closeness principle
  • Definition 7: The family of adversarial game-based protection.
  • Theorem 1: PII guarantee on Crafter
  • ...and 1 more