Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FRIDAY: Mitigating Unintentional Facial Identity in Deepfake Detectors Guided by Facial Recognizers

Younhun Kim, Myung-Joon Kwon, Wonjun Lee, Changick Kim

TL;DR

FRIDAY tackles the generalization gap in deepfake detectors caused by learning facial identity rather than artifacts. It introduces a two-phase training approach where a face recognizer is trained first and then frozen to guide the detector with a Facial Identity Attenuating loss that minimizes embedding similarity: $L_{fia} = \left| \frac{z^f \cdot z^d}{\|z^f\|_2 \|z^d\|_2} \right|$. The detector optimization combines $L_{cls}$ with $\lambda L_{fia}$ to produce $L_{total} = L_{cls} + \lambda L_{fia}$, promoting independence from identity cues. Empirical results on FF++ , Celeb-DF, and DFD show improved ACC and AUC in both in-domain and cross-domain settings, demonstrating stronger generalization and practical robustness.

Abstract

Previous Deepfake detection methods perform well within their training domains, but their effectiveness diminishes significantly with new synthesis techniques. Recent studies have revealed that detection models often create decision boundaries based on facial identity rather than synthetic artifacts, resulting in poor performance on cross-domain datasets. To address this limitation, we propose Facial Recognition Identity Attenuation (FRIDAY), a novel training method that mitigates facial identity influence using a face recognizer. Specifically, we first train a face recognizer using the same backbone as the Deepfake detector. The recognizer is then frozen and employed during the detector's training to reduce facial identity information. This is achieved by feeding input images into both the recognizer and the detector, and minimizing the similarity of their feature embeddings through our Facial Identity Attenuating loss. This process encourages the detector to generate embeddings distinct from the recognizer, effectively reducing the impact of facial identity. Extensive experiments demonstrate that our approach significantly enhances detection performance on both in-domain and cross-domain datasets.

FRIDAY: Mitigating Unintentional Facial Identity in Deepfake Detectors Guided by Facial Recognizers

TL;DR

FRIDAY tackles the generalization gap in deepfake detectors caused by learning facial identity rather than artifacts. It introduces a two-phase training approach where a face recognizer is trained first and then frozen to guide the detector with a Facial Identity Attenuating loss that minimizes embedding similarity: . The detector optimization combines with to produce , promoting independence from identity cues. Empirical results on FF++ , Celeb-DF, and DFD show improved ACC and AUC in both in-domain and cross-domain settings, demonstrating stronger generalization and practical robustness.

Abstract

Previous Deepfake detection methods perform well within their training domains, but their effectiveness diminishes significantly with new synthesis techniques. Recent studies have revealed that detection models often create decision boundaries based on facial identity rather than synthetic artifacts, resulting in poor performance on cross-domain datasets. To address this limitation, we propose Facial Recognition Identity Attenuation (FRIDAY), a novel training method that mitigates facial identity influence using a face recognizer. Specifically, we first train a face recognizer using the same backbone as the Deepfake detector. The recognizer is then frozen and employed during the detector's training to reduce facial identity information. This is achieved by feeding input images into both the recognizer and the detector, and minimizing the similarity of their feature embeddings through our Facial Identity Attenuating loss. This process encourages the detector to generate embeddings distinct from the recognizer, effectively reducing the impact of facial identity. Extensive experiments demonstrate that our approach significantly enhances detection performance on both in-domain and cross-domain datasets.

Paper Structure

This paper contains 16 sections, 9 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Face Recognizer ID-Attenuating Methodology
  3. Motivation
  4. Phase 1: Face Recognizer Training
  5. Phase 2: Deepfake Detector Training
  6. Experiments
  7. Experimental Settings
  8. Datasets
  9. Evaluation Metrics
  10. Evaluation Rules
  11. Implementation Details
  12. Data Preprocessing
  13. Other Details
  14. Results
  15. Effect of $\lambda$
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Overview of FRIDAY. During the Deepfake detector training, unintended facial identity learning occurs, causing biased predictions. We address this by pushing the Deepfake detector's decision boundary away from the face recognizer's, leveraging its identity extraction capability.
  • Figure 2: Demonstrating how much FRIDAY reduces face identity information.
  • Figure 3: FRIDAY Training Process.
  • Figure 4: Effect of $\lambda$. These figures illustrate the effect of varying $\lambda$ on FRIDAY. (a) and (b) depict the accuracy and AUC, both of which are highest at $\lambda=10$. (c) shows the amount of face identity using the unintentional face identity check method referenced in Fig. s\ref{['fig:face_identity_a']}. According to the result, $\lambda=10$ exhibits the lowest inclusion of face identity.