DomainForensics: Exposing Face Forgery across Domains via Bi-directional Adaptation

TL;DR

This paper describes a new bi-directional adaptation strategy dedicated to capturing the forgery knowledge across domains, which is surprisingly effective in exposing new forgeries, and can be plug-and-play on other DeepFake detection architectures.

Abstract

Recent DeepFake detection methods have shown excellent performance on public datasets but are significantly degraded on new forgeries. Solving this problem is important, as new forgeries emerge daily with the continuously evolving generative techniques. Many efforts have been made for this issue by seeking the commonly existing traces empirically on data level. In this paper, we rethink this problem and propose a new solution from the unsupervised domain adaptation perspective. Our solution, called DomainForensics, aims to transfer the forgery knowledge from known forgeries to new forgeries. Unlike recent efforts, our solution does not focus on data view but on learning strategies of DeepFake detectors to capture the knowledge of new forgeries through the alignment of domain discrepancies. In particular, unlike the general domain adaptation methods which consider the knowledge transfer in the semantic class category, thus having limited application, our approach captures the subtle forgery traces. We describe a new bi-directional adaptation strategy dedicated to capturing the forgery knowledge across domains. Specifically, our strategy considers both forward and backward adaptation, to transfer the forgery knowledge from the source domain to the target domain in forward adaptation and then reverse the adaptation from the target domain to the source domain in backward adaptation. In forward adaptation, we perform supervised training for the DeepFake detector in the source domain and jointly employ adversarial feature adaptation to transfer the ability to detect manipulated faces from known forgeries to new forgeries. In backward adaptation, we further improve the knowledge transfer by coupling adversarial adaptation with self-distillation on new forgeries. This enables the detector to expose new forgery features from unlabeled data and avoid forgetting the known knowledge of known...

Figures (7)

• Figure 1: Overview of traditional forensics (top) and DomainForensics (bottom). Traditional forensics achieves excellent performance on known forgeries but performs poorly on new forgeries. In contrast, DomainForensics can effectively expose new forgeries by performing the proposed bi-directional adaption, which can learn the common forgery features across domains using adversarial training.
• Figure 2: Grad-CAM visualization. We train the models, including DANN DANN_JMLR_ODA_5, MDD zhang2019bridging_mdd and our DomainForensics, and visualize the activation maps on FF++ dataset under FS$\rightarrow$F2F scenario. These figures show that models fails to fully capture the common forgery features when only employing one-directional adaptation.
• Figure 3: Illustration of the proposed bi-directional adaptation strategy, containing forward adaptation and backward adaptation with $\mathcal{H}'$ as the final DeepFake detector. Note that other architectures can also be used in our framework.
• Figure 4: Network architecture for DeepFake detector.
• Figure 5: T-SNE van2008visualizing visualization on FF++ (HQ).