Deep Variational Privacy Funnel: General Modeling with Applications in Face Recognition

TL;DR

Problem: protect sensitive attributes in face-recognition embeddings without significantly sacrificing utility. Approach: a Deep Variational Privacy Funnel (DVPF) that learns a stochastic encoder $P_{Z|X}$ to maximize $I(X;Z)$ while constraining $I(S;Z)\le R^{s}$, aided by variational bounds on leakage and utility. Contributions: (i) two tractable objectives (P1, P2) with alternating optimization; (ii) parameterized upper bounds for $I(S;Z)$ and a reconstruction-based bound for $I(X;Z)$; (iii) empirical demonstration on Morph, FairFace, and IJB-C showing leakage reduction and near-random $S$-accuracy at high leakage weight; (iv) a reproducible software package. Significance: enables practical privacy-preserving representations in face-recognition pipelines and provides a framework extendable to privacy-aware deep representation learning.

Abstract

In this study, we harness the information-theoretic Privacy Funnel (PF) model to develop a method for privacy-preserving representation learning using an end-to-end training framework. We rigorously address the trade-off between obfuscation and utility. Both are quantified through the logarithmic loss, a measure also recognized as self-information loss. This exploration deepens the interplay between information-theoretic privacy and representation learning, offering substantive insights into data protection mechanisms for both discriminative and generative models. Importantly, we apply our model to state-of-the-art face recognition systems. The model demonstrates adaptability across diverse inputs, from raw facial images to both derived or refined embeddings, and is competent in tasks such as classification, reconstruction, and generation.

Figures (1)

• Figure 1: Training architecture associated with $\textsf{DVPF}\, (\textsf{P1})$.