Data-Agnostic Face Image Synthesis Detection Using Bayesian CNNs

TL;DR

The paper tackles the problem of detecting face image synthesis without using any synthetic data for training. It introduces a data-agnostic anomaly-detection framework built around a fine-to-coarse Bayesian CNN that learns from real face images and outputs a predictive Gaussian $p(y|z, \mathcal{D}) = \mathcal{N}(y| f(z, w_{MAP}), \sigma^{2}(z))$, with $\sigma^{2}(z)$ defined as $\beta^{-1}+g^{\top}(\alpha I + \beta H)^{-1} g$, enabling posterior-based discrimination between real and unseen synthetic samples. Classification hinges on a posterior threshold $\gamma$, exploiting the empirically higher posteriors for real data. Experiments on FFHQ and CELEBA across four synthesizers (SGAN2, XL-GAN, InsGen, DDPM) show the method is competitive with, and sometimes superior to, state-of-the-art detectors that require synthetic training data, while also highlighting sensitivity to post-processing that erodes small-detail artifacts. The work advances practical detection of synthetic faces by avoiding dependence on disclosed synthesizers and demonstrates a principled, uncertainty-aware approach to one-class recognition in this domain.

Abstract

Face image synthesis detection is considerably gaining attention because of the potential negative impact on society that this type of synthetic data brings. In this paper, we propose a data-agnostic solution to detect the face image synthesis process. Specifically, our solution is based on an anomaly detection framework that requires only real data to learn the inference process. It is therefore data-agnostic in the sense that it requires no synthetic face images. The solution uses the posterior probability with respect to the reference data to determine if new samples are synthetic or not. Our evaluation results using different synthesizers show that our solution is very competitive against the state-of-the-art, which requires synthetic data for training.

Figures (4)

• Figure 1: Sample artifacts in synthetic face images (left to right). Woman with ripples close to the chin, woman with unpaired nostril, man with scar below his left eye, and man with uneven ears.
• Figure 2: Example of the synthetic face images generated by the synthesizers.
• Figure 3: Posterior values produced by the proposed Bayesian CNN. The real and synthetic samples form two distinct regions. Thus, we can set the posterior threshold accordingly.
• Figure 4: Performance or the proposed solution when post-processing is used on the test images.