Detecting Face Synthesis Using a Concealed Fusion Model

TL;DR

Addressing the rising threat of face image synthesis to biometric authentication, the paper develops a detection framework that is resilient to adversarial manipulation. It introduces conceal-features fusion, combining a bank of $K$ models whose outputs are projected with a random polynomial mapping $\rho$ using a secret key to form $\hat{X}_\rho$ of dimension $2K$, then classified by a Bayesian model with two FC layers to yield a real/vs fake score. Key contributions include maintaining a non-quantized feature space, achieving competitive detection accuracy with limited training data, and demonstrating robustness against poisoning, perturbation, backdoor, and reverse-model attacks. The approach enhances practical security for face-validation systems, with future work aimed at cross-dataset evaluation and stronger defenses.

Abstract

Face image synthesis is gaining more attention in computer security due to concerns about its potential negative impacts, including those related to fake biometrics. Hence, building models that can detect the synthesized face images is an important challenge to tackle. In this paper, we propose a fusion-based strategy to detect face image synthesis while providing resiliency to several attacks. The proposed strategy uses a late fusion of the outputs computed by several undisclosed models by relying on random polynomial coefficients and exponents to conceal a new feature space. Unlike existing concealing solutions, our strategy requires no quantization, which helps to preserve the feature space. Our experiments reveal that our strategy achieves state-of-the-art performance while providing protection against poisoning, perturbation, backdoor, and reverse model attacks.

Figures (4)

• Figure 1: Our strategy uses a bank of $K$ models. It projects and encrypts the outputs of the decision layer of each individual model to a new feature space. The encrypted projection is used to train a Bayesian model to classify the samples as real or fake.
• Figure 2: (left) mAp values for several data splits using the 140K sGAN2 images -- the horizontal axis shows the percentage of training data. (right) mAp values on the 140K images of each of the three synthesizers using an 80:20 data split.
• Figure 3: (left) Loss and posterior confidence values during training. (right) Number of attempts needed for the Bayesian fusion model to converge for several key lengths.
• Figure 4: Success rate of several attacks against the proposed Bayesian fusion strategy.