Securing Social Media Against Deepfakes using Identity, Behavioral, and Geometric Signatures

TL;DR

This work tackles the generalization bottleneck in social media deepfake detection by introducing a holistic DBaG descriptor that fuses deep identity, behavioral, and geometric cues. The DBaGNet classifier, trained with a triplet-loss objective, learns robust embeddings that generalize across unseen manipulations and datasets. Extensive cross-dataset and cross-manipulation experiments on six large-scale benchmarks demonstrate state-of-the-art or competitive performance, with ablations confirming the benefit of each feature component. The approach holds practical value for real-world platforms by offering a robust, explainable feature set and a scalable triplet-based detector. Future work aims to incorporate additional cues such as emotion and gaze to further strengthen detection robustness.

Abstract

Trust in social media is a growing concern due to its ability to influence significant societal changes. However, this space is increasingly compromised by various types of deepfake multimedia, which undermine the authenticity of shared content. Although substantial efforts have been made to address the challenge of deepfake content, existing detection techniques face a major limitation in generalization: they tend to perform well only on specific types of deepfakes they were trained on.This dependency on recognizing specific deepfake artifacts makes current methods vulnerable when applied to unseen or varied deepfakes, thereby compromising their performance in real-world applications such as social media platforms. To address the generalizability of deepfake detection, there is a need for a holistic approach that can capture a broader range of facial attributes and manipulations beyond isolated artifacts. To address this, we propose a novel deepfake detection framework featuring an effective feature descriptor that integrates Deep identity, Behavioral, and Geometric (DBaG) signatures, along with a classifier named DBaGNet. Specifically, the DBaGNet classifier utilizes the extracted DBaG signatures, leveraging a triplet loss objective to enhance generalized representation learning for improved classification. Specifically, the DBaGNet classifier utilizes the extracted DBaG signatures and applies a triplet loss objective to enhance generalized representation learning for improved classification. To test the effectiveness and generalizability of our proposed approach, we conduct extensive experiments using six benchmark deepfake datasets: WLDR, CelebDF, DFDC, FaceForensics++, DFD, and NVFAIR. Specifically, to ensure the effectiveness of our approach, we perform cross-dataset evaluations, and the results demonstrate significant performance gains over several state-of-the-art methods.

Figures (5)

• Figure 1: Analysis of in- and cross-dataset representations of individual and fused features. The left column highlights the facial regions where deep identity, behavioral, and geometric features are extracted. The scatter plots display the t-SNE visualization of real and fake samples using different feature types: deep identity in the top row, behavioral in the second, geometric in the third, and the fused DBaG (Deep identity, Behavioral, and Geometric) features in the bottom row. The DBaG fusion demonstrates superior discrimination between real and fake samples, enhancing classification performance and generalization across both within-dataset and cross-dataset tests.
• Figure 2: Visual representation of the proposed feature descriptor, combining (a) deep identity features, (b) behavioral features, and (c) face geometry features to form the comprehensive DBaG descriptor (d).
• Figure 3: Detailed overview of the proposed feature descriptor. In preprocessing step, face detection and cropping are performed followed by feature extraction step where DBaG descriptor composed of Deep Identity, Behavioral and Geometric information is extracted.
• Figure 4: Detailed architecture of the proposed DBaGNet with triplet loss for representation learning.
• Figure 5: Effectiveness of proposed framework on cross-manipulation evaluation: (a) shows the effectiveness of model when trained on DeepFake (DF) and tested on FaceSwap (FS) and Face Shifter (FST) subsets of FF++ and test part of FS. In (b) & (c) model trained on FS and FST and tested on the other two subsets.