Cross-Branch Orthogonality for Improved Generalization in Face Deepfake Detection

TL;DR

The paper tackles the challenge of cross-dataset generalisation in face deepfake detection by introducing CBO-DD, a multi-branch architecture that learns diverse, non-redundant features through an Orthogonal Feature Disentanglement Module enforcing branch-level and cross-branch independence. It combines Localised Spatial, Multi-scale Global Context, and Complementary Emotion branches, with simple concatenation-based fusion made effective by the disentanglement constraints, and a lightweight classifier. Empirical results on FF++, Celeb-DF-V2, and DFDC show strong cross-dataset gains (up to 7-9% in AUC) over state-of-the-art methods without domain adaptation, highlighting improved robustness to unseen manipulations. The approach advances practical deepfake detection by improving generalisation and interpretability through disentangled, orthogonal feature representations.

Abstract

Remarkable advancements in generative AI technology have given rise to a spectrum of novel deepfake categories with unprecedented leaps in their realism, and deepfakes are increasingly becoming a nuisance to law enforcement authorities and the general public. In particular, we observe alarming levels of confusion, deception, and loss of faith regarding multimedia content within society caused by face deepfakes, and existing deepfake detectors are struggling to keep up with the pace of improvements in deepfake generation. This is primarily due to their reliance on specific forgery artifacts, which limits their ability to generalise and detect novel deepfake types. To combat the spread of malicious face deepfakes, this paper proposes a new strategy that leverages coarse-to-fine spatial information, semantic information, and their interactions while ensuring feature distinctiveness and reducing the redundancy of the modelled features. A novel feature orthogonality-based disentanglement strategy is introduced to ensure branch-level and cross-branch feature disentanglement, which allows us to integrate multiple feature vectors without adding complexity to the feature space or compromising generalisation. Comprehensive experiments on three public benchmarks: FaceForensics++, Celeb-DF, and the Deepfake Detection Challenge (DFDC) show that these design choices enable the proposed approach to outperform current state-of-the-art methods by 5% on the Celeb-DF dataset and 7% on the DFDC dataset in a cross-dataset evaluation setting.

Figures (5)

• Figure 1: Fake faces identified by our Cross-Branch Orthogonal DeepFake Detection (CBO-DD) framework on completely unseen deepfake videos generated from the most recent generative AI video generation tools, including OpenAI SORA, RunwayML Gen-2, Adobe Firefly, LTX-Video, Synthesia, and Luma Dream Machine. The results demonstrate the generalisation of our model across different deepfake types as well as different ages, genders, ethnicities, and image characteristics.
• Figure 2: Method overview: We first extract frame-level facial bounding boxes from the input video. The multi-branch encoder module, which consists of a Localised Spatial Feature Branch, a Multi-scale Global Context Branch, and a Complementary Emotion Feature Branch, extracts multiple semantic features from the frame-level inputs. An Orthogonal Feature Disentanglement Module, which uses two projection heads, $P_{Shared}$ and $P_{disentangled}$, and branch-level $L_{\text{branch-ortho}}$ and cross-branch $L_{\text{cross-ortho}}$ Orthogonality losses, enforces branch-level and cross-branch feature disentanglement. Our deepfake classifier module utilises these disentangled features to generate frame-level classifications. Frame-level classifications are aggregated using a majority voting scheme to generate a video-level classification.
• Figure 3: 2D Visualisation of the distribution of the disentangled feature vectors ($F^{LS}_{\text{disentangled}}, F^{MG}_{\text{disentangled}}$, and $F^{CE}_{\text{disentangled}}$) in the cross-dataset evaluation. We indicate embeddings of the FF++ dataset training set as circles, and the DFDC dataset testing set as squares.
• Figure 4: Qualitative results of our CBO-DD model (trained on FF++ dataset) when tested on videos generated by completely unseen GenAI video generation tools (OpenAI SORA, RunwayML Gen-2, Adobe Firefly, LTX-Video, Synthesia, and Luma Dream Machine). We visualise the sample frames from the videos showing the detected face, along with the video-level deepfake detection confidence, which has been generated by averaging the frame-level detection confidence. For additional visualisations, please refer to the supplementary material.
• Figure 5: Visualisation of feature saliency maps derived from three branches, Localised Spatial Branch ($LS$), Multiscale Global ($MG$) context branch, and Complementary Emotion Branch ($CE$), for sample video frames in the DFDC test dataset.