ExposeAnyone: Personalized Audio-to-Expression Diffusion Models Are Robust Zero-Shot Face Forgery Detectors

TL;DR

ExposeAnyone tackles the unseen-manipulation problem in face forgery detection by learning a self-supervised audio-to-expression diffusion model and personalizing it to a subject via adapters. The method pre-trains on unlabeled audio-visual data, refines 3DMM-based facial representations, and authenticates suspected videos through a content-agnostic diffusion reconstruction distance that compares evidence with and without identity conditioning. Across four standard benchmarks, it achieves an average AUC of 95.22% and outperforms prior methods, including on Sora2-generated content where others fail to generalize. The approach also exhibits robustness to common video corruptions, suggesting practicality for real-world deployment in POI-based forgery detection scenarios.

Abstract

Detecting unknown deepfake manipulations remains one of the most challenging problems in face forgery detection. Current state-of-the-art approaches fail to generalize to unseen manipulations, as they primarily rely on supervised training with existing deepfakes or pseudo-fakes, which leads to overfitting to specific forgery patterns. In contrast, self-supervised methods offer greater potential for generalization, but existing work struggles to learn discriminative representations only from self-supervision. In this paper, we propose ExposeAnyone, a fully self-supervised approach based on a diffusion model that generates expression sequences from audio. The key idea is, once the model is personalized to specific subjects using reference sets, it can compute the identity distances between suspected videos and personalized subjects via diffusion reconstruction errors, enabling person-of-interest face forgery detection. Extensive experiments demonstrate that 1) our method outperforms the previous state-of-the-art method by 4.22 percentage points in the average AUC on DF-TIMIT, DFDCP, KoDF, and IDForge datasets, 2) our model is also capable of detecting Sora2-generated videos, where the previous approaches perform poorly, and 3) our method is highly robust to corruptions such as blur and compression, highlighting the applicability in real-world face forgery detection.

Figures (14)

• Figure 1: Our self-supervised face forgery detection approach: We pre-train our audio-to-expression diffusion model on a large-scale, unlabeled video collection. Then, we personalize our pre-trained model on the reference videos of a person of interest (POI) by inserting a subject-specific adapter. Finally, we authenticate suspected videos of POI by the diffusion reconstruction distance.
• Figure 2: ExposeAnyone framework for face forgery detection. (a) We pre-train an audio-to-expression diffusion model to predict the added noise sequence $\boldsymbol\epsilon^{1:L}$ from a noisy expression sequence ${\boldsymbol z}_t^{1:L}$. Then, we personalize the pre-trained model on a specific subject by inserting an adapter token sequence ${\boldsymbol c}^{1:K}$. (b) After personalization, our model can authenticate videos by computing two reconstruction distances w/ and w/o the adapter ${\boldsymbol c}^{1:K}$. (c) Our model is trained in a self-supervised fashion during both pre-training and personalization.
• Figure 3: Robustness to common corruptions on IDForge. Severity levels are defined in DeeperForensics deeperforensics. Our method is highly consistent on the perturbations especially compression that detectors encounter frequently in real-world scenarios.
• Figure 4: $d_{1}$ (AUC = 58.88)
• Figure 5: $d_{2}$ (AUC = 61.34)