UMCL: Unimodal-generated Multimodal Contrastive Learning for Cross-compression-rate Deepfake Detection

TL;DR

This work tackles cross-compression-rate deepfake detection by deriving three compression-robust modalities (rPPG, facial landmarks, and semantic text prompts) from a single video and aligning them with ASA. It introduces Cross-Quality Similarity Learning (CQSL) to ensure consistency across compression levels, and a shared loss that jointly optimizes classification, alignment, and physiological coherence. Extensive experiments on FF++ and other datasets show superior cross-rate, cross-dataset, and cross-manipulation performance, along with strong modality-robustness under degradation and adversarial prompts. The approach offers interpretable cross-modal relationships and boosts practical deployment by eliminating reliance on external multimodal data while maintaining high accuracy in real-world CCR scenarios.

Abstract

In deepfake detection, the varying degrees of compression employed by social media platforms pose significant challenges for model generalization and reliability. Although existing methods have progressed from single-modal to multimodal approaches, they face critical limitations: single-modal methods struggle with feature degradation under data compression in social media streaming, while multimodal approaches require expensive data collection and labeling and suffer from inconsistent modal quality or accessibility in real-world scenarios. To address these challenges, we propose a novel Unimodal-generated Multimodal Contrastive Learning (UMCL) framework for robust cross-compression-rate (CCR) deepfake detection. In the training stage, our approach transforms a single visual modality into three complementary features: compression-robust rPPG signals, temporal landmark dynamics, and semantic embeddings from pre-trained vision-language models. These features are explicitly aligned through an affinity-driven semantic alignment (ASA) strategy, which models inter-modal relationships through affinity matrices and optimizes their consistency through contrastive learning. Subsequently, our cross-quality similarity learning (CQSL) strategy enhances feature robustness across compression rates. Extensive experiments demonstrate that our method achieves superior performance across various compression rates and manipulation types, establishing a new benchmark for robust deepfake detection. Notably, our approach maintains high detection accuracy even when individual features degrade, while providing interpretable insights into feature relationships through explicit alignment.

Figures (9)

• Figure 1: Paradigm comparison between deepfake detection methods. (a) Unimodal methods rely on visual features susceptible to compression artifacts. (b) Traditional multimodal methods require costly data collection and suffer from unequal modality degradation. (c) Our UMCL derives compression-robust multimodal features (rPPG, landmarks, text embeddings) from a single visual source, ensuring consistent modality availability across compression scenarios.
• Figure 2: Modality-wise feature visualization of CPML cpml using t-SNE van2008visualizing.
• Figure 3: Overview of the proposed unimodal-generated multimodal contrastive learning (UMCL) framework for robust cross-compression rate deepfake detection.
• Figure 4: Pipeline of the proposed Affinity-driven Semantic Alignment (ASA) strategy.
• Figure 5: Pipeline of the proposed Cross-quality rPPG Similarity Learning (CQSL) strategy.