Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Supervised Learning for Detecting AI-Generated Faces as Anomalies

Mian Zou, Baosheng Yu, Yibing Zhan, Kede Ma

TL;DR

An anomaly detection method for AI-generated faces is described by leveraging self-supervised learning of camera-intrinsic and face-specific features purely from photographic face images using a Gaussian mixture model.

Abstract

The detection of AI-generated faces is commonly approached as a binary classification task. Nevertheless, the resulting detectors frequently struggle to adapt to novel AI face generators, which evolve rapidly. In this paper, we describe an anomaly detection method for AI-generated faces by leveraging self-supervised learning of camera-intrinsic and face-specific features purely from photographic face images. The success of our method lies in designing a pretext task that trains a feature extractor to rank four ordinal exchangeable image file format (EXIF) tags and classify artificially manipulated face images. Subsequently, we model the learned feature distribution of photographic face images using a Gaussian mixture model. Faces with low likelihoods are flagged as AI-generated. Both quantitative and qualitative experiments validate the effectiveness of our method. Our code is available at \url{https://github.com/MZMMSEC/AIGFD_EXIF.git}.

Self-Supervised Learning for Detecting AI-Generated Faces as Anomalies

TL;DR

An anomaly detection method for AI-generated faces is described by leveraging self-supervised learning of camera-intrinsic and face-specific features purely from photographic face images using a Gaussian mixture model.

Abstract

The detection of AI-generated faces is commonly approached as a binary classification task. Nevertheless, the resulting detectors frequently struggle to adapt to novel AI face generators, which evolve rapidly. In this paper, we describe an anomaly detection method for AI-generated faces by leveraging self-supervised learning of camera-intrinsic and face-specific features purely from photographic face images. The success of our method lies in designing a pretext task that trains a feature extractor to rank four ordinal exchangeable image file format (EXIF) tags and classify artificially manipulated face images. Subsequently, we model the learned feature distribution of photographic face images using a Gaussian mixture model. Faces with low likelihoods are flagged as AI-generated. Both quantitative and qualitative experiments validate the effectiveness of our method. Our code is available at \url{https://github.com/MZMMSEC/AIGFD_EXIF.git}.
Paper Structure (10 sections, 7 equations, 5 figures, 3 tables)

This paper contains 10 sections, 7 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Self-Supervised Representation Learning
  4. GMM Training and Testing
  5. Experiments
  6. Experimental Setups
  7. Main Results
  8. Comparisons with Other Pre-Trained Features
  9. Further Analysis
  10. Conclusion and Discussion

Figures (5)

  • Figure 1: System diagram of the construction of the proposed anomaly detection method for AI-generated faces, including self-supervised representation learning and GMM training. During testing, faces with low likelihoods are identified as AI-generated.
  • Figure 2: Visual examples of artificially manipulated face images by (a) horizontal eye flipping, (b) horizontal mouth flipping, (c) vertical mouth flipping, and (d) global affine transformation, respectively. (e) The original face is also shown as the reference.
  • Figure 3: The process of local face part flipping as a form of artificial face manipulation.
  • Figure 4: Ablation on the number of Gaussian components of the GMM.
  • Figure 5: t-SNE embeddings van2008visualizing of self-supervised features for photographic (in red) and AI-generated (in blue) face images.